Jul 08 19:39:53.736856 ip-10-0-133-133 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 19:39:53.736871 ip-10-0-133-133 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 19:39:53.736918 ip-10-0-133-133 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 19:39:53.737237 ip-10-0-133-133 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 19:40:05.124563 ip-10-0-133-133 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 19:40:05.124577 ip-10-0-133-133 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 6b4827b04c674ec9b72781193c8d86ce -- Jul 08 19:41:44.228083 ip-10-0-133-133 systemd[1]: Starting Kubernetes Kubelet... Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.618757 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622934 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622946 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622952 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622956 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622960 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622963 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:44.947058 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622966 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:44.704540 ip-10-0-133-133 systemd[1]: Started Kubernetes Kubelet. Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622969 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622972 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622975 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622978 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622982 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622985 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622987 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622991 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622994 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622996 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.622999 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623002 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623004 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623007 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623010 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623013 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623015 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623018 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623021 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:44.955891 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623024 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623027 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623030 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623033 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623035 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623038 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623041 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623044 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623047 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623050 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623054 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623058 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623061 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623064 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623066 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623069 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623072 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623074 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623078 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:44.957267 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623081 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623083 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623086 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623088 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623091 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623093 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623095 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623098 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623100 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623103 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623105 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623108 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623111 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623113 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623116 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623118 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623123 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623126 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623129 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:44.958155 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623132 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623135 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623138 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623141 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623144 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623146 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623148 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623151 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623153 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623156 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623158 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623161 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623163 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623166 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623168 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623171 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623174 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623177 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623179 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623182 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:44.959009 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623184 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623187 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623189 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623192 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623194 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623213 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623217 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623221 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623225 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623229 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623232 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623235 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623237 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623240 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623243 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623951 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623958 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623961 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623964 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623967 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:44.960046 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623970 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623973 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623975 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623978 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623981 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623984 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623989 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623992 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623995 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.623998 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624003 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624006 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624009 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624012 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624014 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624017 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624020 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624023 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624025 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624028 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:44.960886 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624031 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624034 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624037 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624039 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624042 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624045 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624047 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624050 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624053 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624056 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624058 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624061 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624064 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624067 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624069 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624072 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624074 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624077 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624079 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624082 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:44.961808 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624084 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624087 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624091 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624094 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624096 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624099 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624101 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624104 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624106 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624109 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624111 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624114 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624116 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624119 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624121 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624124 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624127 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624131 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624135 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:44.962840 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624137 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624140 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624143 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624146 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624148 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624151 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624153 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624155 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624158 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624161 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624164 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624167 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624169 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624171 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624175 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624180 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624183 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624186 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624189 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624192 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:44.963874 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624216 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624219 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624223 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624225 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624228 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624231 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624234 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624237 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624240 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624243 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624245 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624248 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624251 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624253 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624257 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624621 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624626 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624629 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624632 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:44.964901 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624634 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624637 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624639 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624642 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624645 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624647 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624649 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624652 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624656 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624658 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624661 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624663 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624666 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624668 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624671 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624674 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624677 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624679 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624682 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624684 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:44.965938 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624687 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624691 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624694 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624697 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624700 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624703 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624705 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624708 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624711 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624713 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624716 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624718 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624721 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624723 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624726 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624728 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624731 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624734 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624736 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:44.966592 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624739 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624743 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624746 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624748 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624751 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624753 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624756 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624758 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624761 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624763 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624765 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624770 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624773 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624776 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624779 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624781 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624784 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624786 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624789 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:44.967336 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624791 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624794 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624798 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624801 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624803 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624805 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624808 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624812 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624815 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624817 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624820 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624823 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624825 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624828 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624832 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624835 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624838 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624841 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624843 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624846 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:44.967897 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624848 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624851 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624853 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624856 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624859 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624861 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624864 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624866 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624869 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624871 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624873 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624876 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624878 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624881 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.624883 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625253 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625257 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625327 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625334 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625354 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625358 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 19:41:44.968475 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625362 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625366 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625370 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625374 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625377 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625381 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625385 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625388 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625391 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625394 2569 flags.go:64] FLAG: --cgroup-root="" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625397 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625399 2569 flags.go:64] FLAG: --client-ca-file="" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625402 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625405 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625409 2569 flags.go:64] FLAG: --cluster-domain="" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625412 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625415 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625418 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625421 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625425 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625428 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625431 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625434 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625437 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 08 19:41:44.969009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625440 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625442 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625445 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625449 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625453 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625456 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625460 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625463 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625466 2569 flags.go:64] FLAG: --enable-server="true" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625468 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625472 2569 flags.go:64] FLAG: --event-burst="100" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625475 2569 flags.go:64] FLAG: --event-qps="50" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625478 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625483 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625486 2569 flags.go:64] FLAG: --eviction-hard="" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625489 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625492 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625495 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625498 2569 flags.go:64] FLAG: --eviction-soft="" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625500 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625503 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625506 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625509 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625512 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625514 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 19:41:44.969834 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625517 2569 flags.go:64] FLAG: --feature-gates="" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625521 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625524 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625527 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625530 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625533 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625536 2569 flags.go:64] FLAG: --help="false" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625539 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-133-133.ec2.internal" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625542 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625544 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625547 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625552 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625555 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625560 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625562 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625565 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625568 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625571 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625574 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625576 2569 flags.go:64] FLAG: --kube-reserved="" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625580 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625583 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625586 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625589 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 19:41:44.970697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625591 2569 flags.go:64] FLAG: --lock-file="" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625594 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625597 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625600 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625606 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625609 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625611 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625614 2569 flags.go:64] FLAG: --logging-format="text" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625617 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625620 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625623 2569 flags.go:64] FLAG: --manifest-url="" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625626 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625630 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625633 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625636 2569 flags.go:64] FLAG: --max-pods="110" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625639 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625642 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625645 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625648 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625650 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625654 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625658 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625665 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625668 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625671 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 19:41:44.971571 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625674 2569 flags.go:64] FLAG: --pod-cidr="" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625677 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625682 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625685 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625689 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625692 2569 flags.go:64] FLAG: --port="10250" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625695 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625698 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0088dd2dd7aeb16bf" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625701 2569 flags.go:64] FLAG: --qos-reserved="" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625704 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625707 2569 flags.go:64] FLAG: --register-node="true" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625710 2569 flags.go:64] FLAG: --register-with-taints="" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625713 2569 flags.go:64] FLAG: --registry-burst="10" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625716 2569 flags.go:64] FLAG: --registry-qps="5" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625718 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625721 2569 flags.go:64] FLAG: --reserved-memory="" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625725 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625727 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625730 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625733 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625736 2569 flags.go:64] FLAG: --runonce="false" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625739 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625742 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625744 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625747 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 19:41:44.974732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625750 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625753 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625755 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625761 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625764 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625766 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625769 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625772 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625775 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625777 2569 flags.go:64] FLAG: --system-cgroups="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625780 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625786 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625789 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625792 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625795 2569 flags.go:64] FLAG: --tls-min-version="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625798 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625801 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625803 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625806 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625809 2569 flags.go:64] FLAG: --v="2" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625820 2569 flags.go:64] FLAG: --version="false" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625824 2569 flags.go:64] FLAG: --vmodule="" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625828 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.625831 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625938 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:44.975563 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625943 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625946 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625949 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625951 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625954 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625957 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625959 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625962 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625964 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625967 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625971 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625974 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625977 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625980 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625982 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625985 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625987 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625990 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625994 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625996 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:45.067443 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.625999 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626001 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626004 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626006 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626009 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626012 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626014 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626017 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626019 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626022 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626024 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626027 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626029 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626032 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626034 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626036 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626039 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626042 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626044 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:45.068115 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626047 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626049 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626052 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626056 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626059 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626062 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626064 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626067 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626069 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626072 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626074 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626080 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626083 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626086 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626089 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626092 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626095 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626097 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626100 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:45.068681 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626103 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626105 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626109 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626112 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626115 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626118 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626121 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626124 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626127 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626130 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626132 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626135 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626137 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626140 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626142 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626145 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626148 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626151 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626153 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:45.069243 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626156 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626159 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626161 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626163 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626166 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626170 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626173 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626175 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626178 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626180 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626184 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626187 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626190 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626193 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626209 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626213 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626216 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626219 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626221 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626224 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:45.069751 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.626227 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.626728 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.641093 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.641109 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.641935 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.641943 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642017 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642022 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642025 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642029 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642032 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642035 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642037 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642040 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642043 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:45.070327 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642046 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642049 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642051 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642054 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642056 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642059 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642061 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642064 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642068 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642072 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642074 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642077 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642081 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642084 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642087 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642090 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642093 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642096 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:45.070748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642099 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642103 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642107 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642111 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642114 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642117 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642120 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642123 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642126 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642131 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642134 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642137 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642140 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642142 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642145 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642148 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642150 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642153 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642155 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642158 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:45.071286 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642160 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642163 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642165 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642168 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642170 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642173 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642175 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642178 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642181 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642183 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642186 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642189 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642192 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642195 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642212 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642215 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642218 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642220 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642223 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642226 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:45.071810 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642228 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642232 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642235 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642237 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642240 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642243 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642246 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642248 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642251 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642253 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642256 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642259 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642262 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642265 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642267 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642270 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642272 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642275 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642277 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642279 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:45.072417 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642282 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642286 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642289 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642291 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642294 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642296 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642299 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642301 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642304 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642307 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642309 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642312 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.642317 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642434 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642440 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:41:45.072926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642443 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642447 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642450 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642452 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642455 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642458 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642461 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642463 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642466 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642468 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642472 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642475 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642477 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642480 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642482 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642485 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642487 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642491 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642496 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642499 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:41:45.073404 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642502 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642504 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642507 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642509 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642512 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642515 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642517 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642520 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642522 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642525 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642528 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642531 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642534 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642536 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642539 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642542 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642545 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642547 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642550 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:41:45.073940 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642552 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642555 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642558 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642560 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642563 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642566 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642569 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642571 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642574 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642576 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642579 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642582 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642585 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642588 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642590 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642593 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642595 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642597 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642600 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:41:45.074484 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642602 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642605 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642609 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642613 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642616 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642620 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642622 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642625 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642628 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642631 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642634 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642637 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642640 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642643 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642647 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642650 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642652 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642655 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642658 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:41:45.074988 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642661 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642664 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642667 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642669 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642672 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642676 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642679 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642681 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642684 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642686 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642689 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642691 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642694 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642697 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642699 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642702 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642704 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642707 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642711 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:41:45.075523 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:44.642713 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.642719 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.642832 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.644751 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.646830 2569 server.go:1019] "Starting client certificate rotation" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.646934 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.646967 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.649390 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.654797 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.667033 2569 log.go:25] "Validated CRI v1 runtime API" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.671449 2569 log.go:25] "Validated CRI v1 image API" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.672502 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 19:41:45.076062 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.675184 2569 fs.go:135] Filesystem UUIDs: map[4cf762ed-ae60-4091-8be1-4ef3b03d494e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c1761615-ff52-4d4e-ad74-4b4b1a756ced:/dev/nvme0n1p3] Jul 08 19:41:45.076465 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.675217 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 19:41:45.076465 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.678962 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.681023 2569 manager.go:216] Machine: {Timestamp:2026-07-08 19:41:44.679486629 +0000 UTC m=+0.336594102 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099607 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cb6d14613425d74cdf5fca50c7f6b SystemUUID:ec2cb6d1-4613-425d-74cd-f5fca50c7f6b BootID:6b4827b0-4c67-4ec9-b727-81193c8d86ce Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:4c:48:a9:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:4c:48:a9:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:0d:ca:cd:3f:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.681167 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.681289 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.684369 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.684396 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.685255 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.685267 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.685274 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.685886 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.687408 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690261 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690379 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690409 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690434 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690448 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690459 2569 kubelet.go:398] "Adding apiserver pod source" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690472 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 19:41:45.076628 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690855 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.690872 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.693263 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.695491 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.695519 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696502 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696515 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696521 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696526 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696536 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696542 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696550 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696556 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696564 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696581 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696590 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.696596 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.698141 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.698149 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.701213 2569 server.go:1268] "Started kubelet" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.701271 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.702417 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.702458 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.702490 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.702484 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.702600 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.703804 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.704340 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.706519 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.707333 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-133.ec2.internal.18c067e96d84ac2b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-133.ec2.internal,UID:ip-10-0-133-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-133.ec2.internal,},FirstTimestamp:2026-07-08 19:41:44.701168683 +0000 UTC m=+0.358276153,LastTimestamp:2026-07-08 19:41:44.701168683 +0000 UTC m=+0.358276153,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-133.ec2.internal,}" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.712591 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.712597 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.712666 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713137 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713137 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713165 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713289 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713298 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713581 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713597 2569 factory.go:55] Registering systemd factory Jul 08 19:41:45.077823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713607 2569 factory.go:223] Registration of the systemd container factory successfully Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713818 2569 factory.go:153] Registering CRI-O factory Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713833 2569 factory.go:223] Registration of the crio container factory successfully Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713851 2569 factory.go:103] Registering Raw factory Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.713860 2569 manager.go:1196] Started watching for new ooms in manager Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.714245 2569 manager.go:349] Starting recovery of all containers Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.714349 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.714707 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.719834 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.723121 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dpcx6" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.723368 2569 manager.go:354] Recovery completed Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.727136 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.728796 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.728823 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.728833 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.729390 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.729395 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.729398 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.729412 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730146 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730174 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730184 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730599 2569 policy_none.go:49] "None policy: Start" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730611 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730621 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.730879 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dpcx6" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.731439 2569 policy_none.go:47] "Start" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.801887 2569 manager.go:337] "Starting Device Plugin manager" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.801927 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.801940 2569 server.go:87] "Starting device plugin registration server" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.802243 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.802255 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.802355 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.802445 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.802456 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.802949 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.803011 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.851702 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.851741 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.851763 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.851798 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.854267 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.903373 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.904274 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.904309 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.904323 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.904350 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.188013 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.915777 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.915799 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-133.ec2.internal\": node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:44.941887 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.951879 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal"] Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.951965 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.955070 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.955095 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.955108 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.956292 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.956455 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.956481 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.957963 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.957993 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.958007 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.958037 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.958059 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.958070 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.959113 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.959132 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.959798 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.959824 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:44.959833 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.012097 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-133.ec2.internal\" not found" node="ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.014768 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d5b2f68b97376954d14fb66dc3b7019-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal\" (UID: \"5d5b2f68b97376954d14fb66dc3b7019\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.014788 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5b2f68b97376954d14fb66dc3b7019-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal\" (UID: \"5d5b2f68b97376954d14fb66dc3b7019\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.014806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ab0c00c572ee00d9286c0758cbfdbaf-config\") pod \"kube-apiserver-proxy-ip-10-0-133-133.ec2.internal\" (UID: \"6ab0c00c572ee00d9286c0758cbfdbaf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.016961 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-133.ec2.internal\" not found" node="ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.042125 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.115538 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d5b2f68b97376954d14fb66dc3b7019-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal\" (UID: \"5d5b2f68b97376954d14fb66dc3b7019\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.193160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.115576 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5b2f68b97376954d14fb66dc3b7019-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal\" (UID: \"5d5b2f68b97376954d14fb66dc3b7019\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.194002 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.115604 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ab0c00c572ee00d9286c0758cbfdbaf-config\") pod \"kube-apiserver-proxy-ip-10-0-133-133.ec2.internal\" (UID: \"6ab0c00c572ee00d9286c0758cbfdbaf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.194002 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.115649 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5b2f68b97376954d14fb66dc3b7019-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal\" (UID: \"5d5b2f68b97376954d14fb66dc3b7019\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.194002 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.115664 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ab0c00c572ee00d9286c0758cbfdbaf-config\") pod \"kube-apiserver-proxy-ip-10-0-133-133.ec2.internal\" (UID: \"6ab0c00c572ee00d9286c0758cbfdbaf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.194002 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.115655 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d5b2f68b97376954d14fb66dc3b7019-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal\" (UID: \"5d5b2f68b97376954d14fb66dc3b7019\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.194002 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.142630 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.299918 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.243489 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.410339 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.314723 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.410339 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.320380 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.410339 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.343704 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.528648 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.444253 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.623348 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.544704 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.623348 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.590551 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.645022 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.646097 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.646245 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.646259 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.732748 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 19:36:44 +0000 UTC" deadline="2028-04-15 15:35:44.19246815 +0000 UTC" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.732774 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15523h53m58.45969649s" Jul 08 19:41:45.759347 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.745860 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.858220 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:45.846040 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-133.ec2.internal\" not found" Jul 08 19:41:45.926250 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.886567 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:45.926250 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.912697 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.926250 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.914557 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:45.929095 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.928664 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:41:45.930936 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.929683 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" Jul 08 19:41:45.938237 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:45.937355 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:41:46.202031 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:46.154348 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5b2f68b97376954d14fb66dc3b7019.slice/crio-8b37dfbfe7b3b4b846fc5d7f43fb653ff5c3342d1a650832a0ec02ae2c58950e WatchSource:0}: Error finding container 8b37dfbfe7b3b4b846fc5d7f43fb653ff5c3342d1a650832a0ec02ae2c58950e: Status 404 returned error can't find the container with id 8b37dfbfe7b3b4b846fc5d7f43fb653ff5c3342d1a650832a0ec02ae2c58950e Jul 08 19:41:46.202031 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.160002 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:41:46.327071 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:46.283158 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab0c00c572ee00d9286c0758cbfdbaf.slice/crio-9c74303605db470e515af45cd607d5be563bebe435a06a44a1341918920782b7 WatchSource:0}: Error finding container 9c74303605db470e515af45cd607d5be563bebe435a06a44a1341918920782b7: Status 404 returned error can't find the container with id 9c74303605db470e515af45cd607d5be563bebe435a06a44a1341918920782b7 Jul 08 19:41:46.710087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.692887 2569 apiserver.go:52] "Watching apiserver" Jul 08 19:41:46.710087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.701973 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 19:41:46.710087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.703143 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-dbwfh","kube-system/global-pull-secret-syncer-rdnjw","kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal","openshift-image-registry/node-ca-ppvcz","openshift-multus/multus-x6qqv","openshift-multus/network-metrics-daemon-dvcrj","openshift-ovn-kubernetes/ovnkube-node-92tws","kube-system/konnectivity-agent-85ltm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6","openshift-cluster-node-tuning-operator/tuned-7q57x","openshift-dns/node-resolver-6phpd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal","openshift-multus/multus-additional-cni-plugins-jdhtx","openshift-network-diagnostics/network-check-target-rkvt6"] Jul 08 19:41:46.710087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.706171 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.710087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.708485 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.710087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.709070 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.710194 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.710376 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.710425 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.710572 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.710652 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rg4h2\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.711607 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.711811 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.711987 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dxlr6\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.712391 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.712408 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.712617 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.712467 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.712757 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.713067 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bjl5c\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.713343 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.713513 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.713744 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.713953 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.714130 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.714309 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.714330 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.714574 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.714809 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kdg82\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.714927 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.715313 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.715613 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.715649 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p6brb\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.715970 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.716169 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.716197 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.716444 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-grcl8\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.716521 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.717216 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.717426 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.717689 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b2pmd\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.718023 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.724576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.718224 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.718456 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.718514 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.718561 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.719114 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-czlt8\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.719308 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.719495 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.719872 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.720931 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.720975 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.722218 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.722483 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v42gv\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.722662 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.724904 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-systemd\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.724925 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-cni-bin\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.724943 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-socket-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.724969 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6974\" (UniqueName: \"kubernetes.io/projected/54299b5b-a74c-4b21-83c0-082100f8580b-kube-api-access-g6974\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.724985 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-multus-certs\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725013 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-sys-fs\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725046 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-node-log\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725080 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-env-overrides\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725096 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-system-cni-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.726765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725111 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-cni-multus\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725127 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-hostroot\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725150 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-ovn\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mll7m\" (UniqueName: \"kubernetes.io/projected/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-kube-api-access-mll7m\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725300 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9f78\" (UniqueName: \"kubernetes.io/projected/bfa58b99-1c26-4411-bc9e-25a68ec9e357-kube-api-access-d9f78\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725323 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5v6\" (UniqueName: \"kubernetes.io/projected/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-kube-api-access-cn5v6\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725350 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-cnibin\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725384 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-slash\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725405 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-cni-netd\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725444 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-os-release\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-kubelet\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725495 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-etc-kubernetes\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725520 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-registration-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725548 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-kubelet\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725573 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovn-node-metrics-cert\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725601 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-systemd-units\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.727853 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9444ef70-a6fa-4055-9643-60bff9cac87b-cni-binary-copy\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725837 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-socket-dir-parent\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725883 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-device-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725955 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/54299b5b-a74c-4b21-83c0-082100f8580b-iptables-alerter-script\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.725983 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-run-ovn-kubernetes\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726021 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8ts9\" (UniqueName: \"kubernetes.io/projected/39a4d2a3-681f-4b52-ba42-7f32b11189bb-kube-api-access-h8ts9\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726050 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-serviceca\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-k8s-cni-cncf-io\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726132 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-netns\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx927\" (UniqueName: \"kubernetes.io/projected/9444ef70-a6fa-4055-9643-60bff9cac87b-kube-api-access-lx927\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726243 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-var-lib-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726248 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726269 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovnkube-config\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726303 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-cni-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726336 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-cni-bin\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726384 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54299b5b-a74c-4b21-83c0-082100f8580b-host-slash\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726427 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-etc-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.729046 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726446 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726524 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-hosts-file\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726546 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-log-socket\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726568 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovnkube-script-lib\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726596 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-host\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726637 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-conf-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb8a42e5-d264-4b5b-b877-0454af519fd4-agent-certs\") pod \"konnectivity-agent-85ltm\" (UID: \"eb8a42e5-d264-4b5b-b877-0454af519fd4\") " pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726709 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-etc-selinux\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726769 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-run-netns\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726813 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726841 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb8a42e5-d264-4b5b-b877-0454af519fd4-konnectivity-ca\") pod \"konnectivity-agent-85ltm\" (UID: \"eb8a42e5-d264-4b5b-b877-0454af519fd4\") " pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726868 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726894 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-daemon-config\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726929 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlplf\" (UniqueName: \"kubernetes.io/projected/42b16c47-efdc-4d44-9501-99711b3e6487-kube-api-access-mlplf\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:46.730032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.726959 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-tmp-dir\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.746468 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.746446 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v8gdc" Jul 08 19:41:46.754635 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.754617 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v8gdc" Jul 08 19:41:46.814771 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.814703 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 19:41:46.816419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.816399 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:46.827671 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f63e44-06f8-4da0-8458-54e1c0afff73-tmp\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.827782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827678 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-kubelet\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.827782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827702 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovn-node-metrics-cert\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.827782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827726 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-systemd-units\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.827782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827749 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9444ef70-a6fa-4055-9643-60bff9cac87b-cni-binary-copy\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.827782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827764 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-kubelet\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.827782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827772 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-socket-dir-parent\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-device-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827818 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-modprobe-d\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/54299b5b-a74c-4b21-83c0-082100f8580b-iptables-alerter-script\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827869 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-run-ovn-kubernetes\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827892 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8ts9\" (UniqueName: \"kubernetes.io/projected/39a4d2a3-681f-4b52-ba42-7f32b11189bb-kube-api-access-h8ts9\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827896 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-device-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827914 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-serviceca\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827940 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-run-ovn-kubernetes\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-k8s-cni-cncf-io\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827964 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-socket-dir-parent\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827984 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-netns\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828027 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-k8s-cni-cncf-io\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828029 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-netns\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.827896 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-systemd-units\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828094 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828080 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx927\" (UniqueName: \"kubernetes.io/projected/9444ef70-a6fa-4055-9643-60bff9cac87b-kube-api-access-lx927\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828109 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-lib-modules\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828136 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-var-lib-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828164 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovnkube-config\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828198 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-cni-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828197 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828275 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-var-lib-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828284 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-cni-bin\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828330 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54299b5b-a74c-4b21-83c0-082100f8580b-host-slash\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828361 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-etc-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828420 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysconfig\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828440 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-cni-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828445 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysctl-d\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828442 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-serviceca\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828331 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-cni-bin\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828485 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9444ef70-a6fa-4055-9643-60bff9cac87b-cni-binary-copy\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828500 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-etc-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.828767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-systemd\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828526 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/54299b5b-a74c-4b21-83c0-082100f8580b-iptables-alerter-script\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828553 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828502 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54299b5b-a74c-4b21-83c0-082100f8580b-host-slash\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828565 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828603 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-hosts-file\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-log-socket\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828675 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-hosts-file\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828691 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovnkube-script-lib\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828723 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-log-socket\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-host\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828782 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-conf-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828809 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb8a42e5-d264-4b5b-b877-0454af519fd4-agent-certs\") pod \"konnectivity-agent-85ltm\" (UID: \"eb8a42e5-d264-4b5b-b877-0454af519fd4\") " pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828829 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-host\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828837 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-etc-selinux\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828866 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-kubernetes\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828878 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-conf-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.829573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828915 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-run-netns\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828920 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovnkube-config\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828971 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.828979 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-etc-selinux\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829005 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb8a42e5-d264-4b5b-b877-0454af519fd4-konnectivity-ca\") pod \"konnectivity-agent-85ltm\" (UID: \"eb8a42e5-d264-4b5b-b877-0454af519fd4\") " pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829043 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-openvswitch\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829051 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysctl-conf\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829078 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx5dk\" (UniqueName: \"kubernetes.io/projected/b0f63e44-06f8-4da0-8458-54e1c0afff73-kube-api-access-bx5dk\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829085 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-run-netns\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829092 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovnkube-script-lib\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829120 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-cnibin\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829156 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-tuned\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-daemon-config\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829410 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlplf\" (UniqueName: \"kubernetes.io/projected/42b16c47-efdc-4d44-9501-99711b3e6487-kube-api-access-mlplf\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.829414 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:46.830373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829436 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-run\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829462 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be2ea6fc-bd9b-4078-9634-1c31078e4661-dbus\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829497 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-tmp-dir\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.829531 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:47.329508348 +0000 UTC m=+2.986615821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829569 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-systemd\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829526 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb8a42e5-d264-4b5b-b877-0454af519fd4-konnectivity-ca\") pod \"konnectivity-agent-85ltm\" (UID: \"eb8a42e5-d264-4b5b-b877-0454af519fd4\") " pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829594 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-cni-bin\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829661 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-host\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829676 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-systemd\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829734 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829780 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-socket-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829791 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-tmp-dir\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-var-lib-kubelet\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829821 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-cni-bin\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829836 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6974\" (UniqueName: \"kubernetes.io/projected/54299b5b-a74c-4b21-83c0-082100f8580b-kube-api-access-g6974\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.831132 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829920 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-socket-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-multus-certs\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.829981 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-sys-fs\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830022 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830054 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-run-multus-certs\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-system-cni-dir\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830152 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kgfn\" (UniqueName: \"kubernetes.io/projected/9ccc8460-155d-4388-8080-8d3c458aa401-kube-api-access-5kgfn\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830249 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9444ef70-a6fa-4055-9643-60bff9cac87b-multus-daemon-config\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830253 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-node-log\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830287 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-env-overrides\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830312 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-node-log\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830314 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-system-cni-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830359 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-cni-multus\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830384 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-hostroot\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830402 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-system-cni-dir\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830410 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be2ea6fc-bd9b-4078-9634-1c31078e4661-kubelet-config\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.831622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830454 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-cni-multus\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830456 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-hostroot\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830475 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-ovn\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830501 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mll7m\" (UniqueName: \"kubernetes.io/projected/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-kube-api-access-mll7m\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830527 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830533 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-run-ovn\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830556 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9f78\" (UniqueName: \"kubernetes.io/projected/bfa58b99-1c26-4411-bc9e-25a68ec9e357-kube-api-access-d9f78\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830594 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-sys\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830609 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830623 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-os-release\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830655 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5v6\" (UniqueName: \"kubernetes.io/projected/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-kube-api-access-cn5v6\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830685 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-cnibin\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-slash\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830720 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39a4d2a3-681f-4b52-ba42-7f32b11189bb-env-overrides\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-cni-netd\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830773 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-os-release\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830787 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-slash\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830798 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-kubelet\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830808 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-sys-fs\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830864 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-etc-kubernetes\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830810 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-cnibin\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830853 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-host-var-lib-kubelet\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830859 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39a4d2a3-681f-4b52-ba42-7f32b11189bb-host-cni-netd\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830859 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-os-release\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830828 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9444ef70-a6fa-4055-9643-60bff9cac87b-etc-kubernetes\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.830940 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-registration-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.831008 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfa58b99-1c26-4411-bc9e-25a68ec9e357-registration-dir\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.832065 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39a4d2a3-681f-4b52-ba42-7f32b11189bb-ovn-node-metrics-cert\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.832642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.832144 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb8a42e5-d264-4b5b-b877-0454af519fd4-agent-certs\") pod \"konnectivity-agent-85ltm\" (UID: \"eb8a42e5-d264-4b5b-b877-0454af519fd4\") " pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:46.848458 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.848432 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mll7m\" (UniqueName: \"kubernetes.io/projected/9a05fefd-7dcf-4fdc-833e-6ee404a4de0f-kube-api-access-mll7m\") pod \"node-ca-ppvcz\" (UID: \"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f\") " pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:46.848635 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.848616 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8ts9\" (UniqueName: \"kubernetes.io/projected/39a4d2a3-681f-4b52-ba42-7f32b11189bb-kube-api-access-h8ts9\") pod \"ovnkube-node-92tws\" (UID: \"39a4d2a3-681f-4b52-ba42-7f32b11189bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:46.848677 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.848620 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9f78\" (UniqueName: \"kubernetes.io/projected/bfa58b99-1c26-4411-bc9e-25a68ec9e357-kube-api-access-d9f78\") pod \"aws-ebs-csi-driver-node-btxz6\" (UID: \"bfa58b99-1c26-4411-bc9e-25a68ec9e357\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:46.848815 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.848796 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx927\" (UniqueName: \"kubernetes.io/projected/9444ef70-a6fa-4055-9643-60bff9cac87b-kube-api-access-lx927\") pod \"multus-x6qqv\" (UID: \"9444ef70-a6fa-4055-9643-60bff9cac87b\") " pod="openshift-multus/multus-x6qqv" Jul 08 19:41:46.848917 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.848903 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5v6\" (UniqueName: \"kubernetes.io/projected/3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84-kube-api-access-cn5v6\") pod \"node-resolver-6phpd\" (UID: \"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84\") " pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:46.849231 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.849217 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlplf\" (UniqueName: \"kubernetes.io/projected/42b16c47-efdc-4d44-9501-99711b3e6487-kube-api-access-mlplf\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:46.849271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.849246 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6974\" (UniqueName: \"kubernetes.io/projected/54299b5b-a74c-4b21-83c0-082100f8580b-kube-api-access-g6974\") pod \"iptables-alerter-dbwfh\" (UID: \"54299b5b-a74c-4b21-83c0-082100f8580b\") " pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:46.855908 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.855865 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" event={"ID":"6ab0c00c572ee00d9286c0758cbfdbaf","Type":"ContainerStarted","Data":"9c74303605db470e515af45cd607d5be563bebe435a06a44a1341918920782b7"} Jul 08 19:41:46.856875 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.856854 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" event={"ID":"5d5b2f68b97376954d14fb66dc3b7019","Type":"ContainerStarted","Data":"8b37dfbfe7b3b4b846fc5d7f43fb653ff5c3342d1a650832a0ec02ae2c58950e"} Jul 08 19:41:46.932011 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.931974 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-tuned\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932011 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932010 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932032 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-run\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932053 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be2ea6fc-bd9b-4078-9634-1c31078e4661-dbus\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932075 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-host\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932098 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932123 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-var-lib-kubelet\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932163 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be2ea6fc-bd9b-4078-9634-1c31078e4661-dbus\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932142 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-run\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932172 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.932257 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932255 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-var-lib-kubelet\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-system-cni-dir\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932143 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-host\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932302 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.932324 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:47.432306731 +0000 UTC m=+3.089414204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932326 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-system-cni-dir\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932356 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kgfn\" (UniqueName: \"kubernetes.io/projected/9ccc8460-155d-4388-8080-8d3c458aa401-kube-api-access-5kgfn\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932391 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be2ea6fc-bd9b-4078-9634-1c31078e4661-kubelet-config\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932431 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-sys\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932453 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-os-release\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932490 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f63e44-06f8-4da0-8458-54e1c0afff73-tmp\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932519 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-modprobe-d\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932555 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-sys\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932558 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-lib-modules\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932595 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysconfig\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysctl-d\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932640 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-systemd\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.932836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932662 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932675 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-lib-modules\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932692 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-kubernetes\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932697 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932719 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysctl-conf\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932742 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx5dk\" (UniqueName: \"kubernetes.io/projected/b0f63e44-06f8-4da0-8458-54e1c0afff73-kube-api-access-bx5dk\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932749 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932763 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-cnibin\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932757 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-modprobe-d\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932797 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysctl-d\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932811 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-kubernetes\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932761 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ccc8460-155d-4388-8080-8d3c458aa401-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932838 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysconfig\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932839 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be2ea6fc-bd9b-4078-9634-1c31078e4661-kubelet-config\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932849 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-os-release\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932874 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-cnibin\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932883 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ccc8460-155d-4388-8080-8d3c458aa401-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.933376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932949 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-sysctl-conf\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.933859 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.932997 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-systemd\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.934284 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.934257 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0f63e44-06f8-4da0-8458-54e1c0afff73-etc-tuned\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.934678 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.934658 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f63e44-06f8-4da0-8458-54e1c0afff73-tmp\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:46.943494 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.943468 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:41:46.943494 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.943489 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:41:46.943655 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.943502 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:46.943655 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:46.943576 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:41:47.443559584 +0000 UTC m=+3.100667061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:46.945602 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.945578 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kgfn\" (UniqueName: \"kubernetes.io/projected/9ccc8460-155d-4388-8080-8d3c458aa401-kube-api-access-5kgfn\") pod \"multus-additional-cni-plugins-jdhtx\" (UID: \"9ccc8460-155d-4388-8080-8d3c458aa401\") " pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:46.946544 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:46.946524 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx5dk\" (UniqueName: \"kubernetes.io/projected/b0f63e44-06f8-4da0-8458-54e1c0afff73-kube-api-access-bx5dk\") pod \"tuned-7q57x\" (UID: \"b0f63e44-06f8-4da0-8458-54e1c0afff73\") " pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:47.025608 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.025569 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dbwfh" Jul 08 19:41:47.033641 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.033451 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6phpd" Jul 08 19:41:47.034436 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.034404 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54299b5b_a74c_4b21_83c0_082100f8580b.slice/crio-fe0156218fffb415aa795346ad296bc88788bd4f530d81747598e613cd72c45a WatchSource:0}: Error finding container fe0156218fffb415aa795346ad296bc88788bd4f530d81747598e613cd72c45a: Status 404 returned error can't find the container with id fe0156218fffb415aa795346ad296bc88788bd4f530d81747598e613cd72c45a Jul 08 19:41:47.042029 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.042000 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e17ecd3_1b9f_4cc2_8df5_d6ba168c3e84.slice/crio-f6b591490463e866133af570b3ecd902c21d5b6001abcf8636765429cc5a1a6c WatchSource:0}: Error finding container f6b591490463e866133af570b3ecd902c21d5b6001abcf8636765429cc5a1a6c: Status 404 returned error can't find the container with id f6b591490463e866133af570b3ecd902c21d5b6001abcf8636765429cc5a1a6c Jul 08 19:41:47.049103 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.049079 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:41:47.053955 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.053934 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ppvcz" Jul 08 19:41:47.060639 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.060605 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a4d2a3_681f_4b52_ba42_7f32b11189bb.slice/crio-e1edb060e18a991aef0ea8320f52e3ca62b5b97b812f0d6e26bbea5e012d2717 WatchSource:0}: Error finding container e1edb060e18a991aef0ea8320f52e3ca62b5b97b812f0d6e26bbea5e012d2717: Status 404 returned error can't find the container with id e1edb060e18a991aef0ea8320f52e3ca62b5b97b812f0d6e26bbea5e012d2717 Jul 08 19:41:47.060743 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.060697 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x6qqv" Jul 08 19:41:47.067127 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.067100 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a05fefd_7dcf_4fdc_833e_6ee404a4de0f.slice/crio-12bed236739d63449676a9fc49fac5cb73a539f94b88a12037757a888ac5c9ac WatchSource:0}: Error finding container 12bed236739d63449676a9fc49fac5cb73a539f94b88a12037757a888ac5c9ac: Status 404 returned error can't find the container with id 12bed236739d63449676a9fc49fac5cb73a539f94b88a12037757a888ac5c9ac Jul 08 19:41:47.067241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.067146 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:41:47.071926 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.071750 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9444ef70_a6fa_4055_9643_60bff9cac87b.slice/crio-a65b655b73cdd1cfe12117e60fcd11a2a106b8722521020b3f4424174b841d1b WatchSource:0}: Error finding container a65b655b73cdd1cfe12117e60fcd11a2a106b8722521020b3f4424174b841d1b: Status 404 returned error can't find the container with id a65b655b73cdd1cfe12117e60fcd11a2a106b8722521020b3f4424174b841d1b Jul 08 19:41:47.077318 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.077295 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" Jul 08 19:41:47.077997 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.077974 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8a42e5_d264_4b5b_b877_0454af519fd4.slice/crio-d9edd42ee1963957ecdf0bd6036649cc4b70be5a4b906724a559a46886b132a0 WatchSource:0}: Error finding container d9edd42ee1963957ecdf0bd6036649cc4b70be5a4b906724a559a46886b132a0: Status 404 returned error can't find the container with id d9edd42ee1963957ecdf0bd6036649cc4b70be5a4b906724a559a46886b132a0 Jul 08 19:41:47.082697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.082555 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7q57x" Jul 08 19:41:47.087340 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.087315 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa58b99_1c26_4411_bc9e_25a68ec9e357.slice/crio-179eefd58907cdcc1e73e5d31d5057cdc825b7871e544b638c875d49a306a4d5 WatchSource:0}: Error finding container 179eefd58907cdcc1e73e5d31d5057cdc825b7871e544b638c875d49a306a4d5: Status 404 returned error can't find the container with id 179eefd58907cdcc1e73e5d31d5057cdc825b7871e544b638c875d49a306a4d5 Jul 08 19:41:47.088081 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.088063 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" Jul 08 19:41:47.091970 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.091940 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f63e44_06f8_4da0_8458_54e1c0afff73.slice/crio-975f7fc62d7631ab8b26ae31b9c4dc97bdf50f096baa585b8f394112adccc735 WatchSource:0}: Error finding container 975f7fc62d7631ab8b26ae31b9c4dc97bdf50f096baa585b8f394112adccc735: Status 404 returned error can't find the container with id 975f7fc62d7631ab8b26ae31b9c4dc97bdf50f096baa585b8f394112adccc735 Jul 08 19:41:47.099500 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:41:47.099471 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ccc8460_155d_4388_8080_8d3c458aa401.slice/crio-79035b3223bd17ae67ecdbf1458768028924dc4cbecab3319ea316176f666e4a WatchSource:0}: Error finding container 79035b3223bd17ae67ecdbf1458768028924dc4cbecab3319ea316176f666e4a: Status 404 returned error can't find the container with id 79035b3223bd17ae67ecdbf1458768028924dc4cbecab3319ea316176f666e4a Jul 08 19:41:47.335820 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.335727 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:47.336587 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.335983 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:47.336587 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.336058 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:48.336040485 +0000 UTC m=+3.993147945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:47.436257 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.436224 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:47.436419 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.436361 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:47.436471 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.436421 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:48.436408215 +0000 UTC m=+4.093515676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:47.537419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.537383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:47.537593 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.537571 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:41:47.537668 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.537601 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:41:47.537668 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.537613 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:47.537763 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:47.537672 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:41:48.537658497 +0000 UTC m=+4.194765955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:47.797540 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.797447 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:36:46 +0000 UTC" deadline="2027-12-11 20:26:18.429158798 +0000 UTC" Jul 08 19:41:47.797540 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.797490 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12504h44m30.631673292s" Jul 08 19:41:47.860844 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.860804 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerStarted","Data":"79035b3223bd17ae67ecdbf1458768028924dc4cbecab3319ea316176f666e4a"} Jul 08 19:41:47.862761 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.862730 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" event={"ID":"bfa58b99-1c26-4411-bc9e-25a68ec9e357","Type":"ContainerStarted","Data":"179eefd58907cdcc1e73e5d31d5057cdc825b7871e544b638c875d49a306a4d5"} Jul 08 19:41:47.864052 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.864027 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-85ltm" event={"ID":"eb8a42e5-d264-4b5b-b877-0454af519fd4","Type":"ContainerStarted","Data":"d9edd42ee1963957ecdf0bd6036649cc4b70be5a4b906724a559a46886b132a0"} Jul 08 19:41:47.867965 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.867884 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ppvcz" event={"ID":"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f","Type":"ContainerStarted","Data":"12bed236739d63449676a9fc49fac5cb73a539f94b88a12037757a888ac5c9ac"} Jul 08 19:41:47.869845 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.869738 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"e1edb060e18a991aef0ea8320f52e3ca62b5b97b812f0d6e26bbea5e012d2717"} Jul 08 19:41:47.872805 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.872783 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6phpd" event={"ID":"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84","Type":"ContainerStarted","Data":"f6b591490463e866133af570b3ecd902c21d5b6001abcf8636765429cc5a1a6c"} Jul 08 19:41:47.874286 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.874262 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7q57x" event={"ID":"b0f63e44-06f8-4da0-8458-54e1c0afff73","Type":"ContainerStarted","Data":"975f7fc62d7631ab8b26ae31b9c4dc97bdf50f096baa585b8f394112adccc735"} Jul 08 19:41:47.875642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.875617 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x6qqv" event={"ID":"9444ef70-a6fa-4055-9643-60bff9cac87b","Type":"ContainerStarted","Data":"a65b655b73cdd1cfe12117e60fcd11a2a106b8722521020b3f4424174b841d1b"} Jul 08 19:41:47.876855 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.876831 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dbwfh" event={"ID":"54299b5b-a74c-4b21-83c0-082100f8580b","Type":"ContainerStarted","Data":"fe0156218fffb415aa795346ad296bc88788bd4f530d81747598e613cd72c45a"} Jul 08 19:41:47.926704 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:47.926474 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:41:48.345238 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.344322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:48.345238 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.344541 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:48.345238 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.344607 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:50.344588966 +0000 UTC m=+6.001696426 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:48.444790 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.444753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:48.444973 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.444912 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:48.445029 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.444982 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:50.444963842 +0000 UTC m=+6.102071314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:48.545839 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.545169 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:48.545839 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.545381 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:41:48.545839 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.545401 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:41:48.545839 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.545413 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:48.545839 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.545473 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:41:50.545455427 +0000 UTC m=+6.202562889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:48.797927 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.797838 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:36:46 +0000 UTC" deadline="2028-04-14 10:54:42.785367092 +0000 UTC" Jul 08 19:41:48.797927 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.797879 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15495h12m53.98749212s" Jul 08 19:41:48.855196 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.854590 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:48.855196 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.854714 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:48.855196 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.854870 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:48.855196 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:48.854888 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:48.855196 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.854946 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:48.855196 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:48.855057 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:50.369624 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:50.369537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:50.370221 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.369705 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:50.370221 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.369784 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:54.369763373 +0000 UTC m=+10.026870834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:50.470879 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:50.470841 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:50.471043 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.470983 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:50.471116 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.471044 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:41:54.471027284 +0000 UTC m=+10.128134741 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:50.571369 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:50.571334 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:50.571533 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.571510 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:41:50.571603 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.571535 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:41:50.571603 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.571547 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:50.571709 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.571611 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:41:54.571590203 +0000 UTC m=+10.228697669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:50.853114 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:50.853025 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:50.853274 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.853186 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:50.853757 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:50.853611 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:50.853757 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.853717 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:50.854130 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:50.853994 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:50.854130 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:50.854092 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:52.853270 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:52.853236 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:52.853740 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:52.853241 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:52.853740 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:52.853548 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:52.853740 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:52.853400 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:52.853740 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:52.853660 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:52.853740 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:52.853373 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:54.402313 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:54.402262 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:54.402717 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.402451 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:54.402717 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.402523 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:02.402508753 +0000 UTC m=+18.059616211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:41:54.503931 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:54.503892 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:54.504093 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.504046 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:54.504151 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.504113 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:02.504100287 +0000 UTC m=+18.161207744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:41:54.605310 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:54.605254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:54.605500 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.605475 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:41:54.605555 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.605514 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:41:54.605555 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.605530 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:54.605616 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.605606 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:42:02.605584375 +0000 UTC m=+18.262691856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:41:54.858419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:54.858343 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:54.858419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:54.858343 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:54.858419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:54.858353 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:54.858675 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.858478 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:54.858675 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.858568 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:54.858781 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:54.858680 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:56.852789 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:56.852758 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:56.853246 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:56.852880 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:56.853246 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:56.852884 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:56.853246 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:56.852768 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:56.853246 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:56.852990 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:56.853246 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:56.853091 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:57.901729 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:57.901523 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" event={"ID":"6ab0c00c572ee00d9286c0758cbfdbaf","Type":"ContainerStarted","Data":"f98385bc5661704101dcdffb450aadff226f4aea26ecb7a773ac537eb826d4c2"} Jul 08 19:41:57.903168 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:57.903138 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7q57x" event={"ID":"b0f63e44-06f8-4da0-8458-54e1c0afff73","Type":"ContainerStarted","Data":"cd514cafaa3c99f4ae1ea9825f9a6d97a77a5f5c9756676f95f6cb8cebd80e7b"} Jul 08 19:41:57.917119 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:57.917061 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-133.ec2.internal" podStartSLOduration=12.917043849 podStartE2EDuration="12.917043849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:57.916609493 +0000 UTC m=+13.573716974" watchObservedRunningTime="2026-07-08 19:41:57.917043849 +0000 UTC m=+13.574151324" Jul 08 19:41:57.938376 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:57.938332 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7q57x" podStartSLOduration=3.02693577 podStartE2EDuration="12.938317363s" totalImagesPullingTime="9.911381593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:57.938191628 +0000 UTC m=+13.595299134" watchObservedRunningTime="2026-07-08 19:41:57.938317363 +0000 UTC m=+13.595424848" Jul 08 19:41:58.853176 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.853047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:41:58.853377 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:58.853175 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:41:58.853377 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.853250 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:41:58.853592 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:58.853371 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:41:58.853592 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.853429 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:41:58.853592 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:41:58.853498 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:41:58.907590 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.907548 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6phpd" event={"ID":"3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84","Type":"ContainerStarted","Data":"7350fc09d0797dfc0cdb6bf8131a4c31e282c5dbe429782c6852339bc40f2606"} Jul 08 19:41:58.910999 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.910611 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dbwfh" event={"ID":"54299b5b-a74c-4b21-83c0-082100f8580b","Type":"ContainerStarted","Data":"ee4ba112cdd298dbcdd66207713f389704dcf89ba62030e4b362a81d91081793"} Jul 08 19:41:58.913942 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.913685 2569 generic.go:356] "Generic (PLEG): container finished" podID="5d5b2f68b97376954d14fb66dc3b7019" containerID="1e109fa7248be859730f4c0514b9809c5dcc7e789b454ead2fa6d6daed9ec7f4" exitCode=0 Jul 08 19:41:58.914428 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.914145 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" event={"ID":"5d5b2f68b97376954d14fb66dc3b7019","Type":"ContainerDied","Data":"1e109fa7248be859730f4c0514b9809c5dcc7e789b454ead2fa6d6daed9ec7f4"} Jul 08 19:41:58.917171 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.917045 2569 generic.go:356] "Generic (PLEG): container finished" podID="9ccc8460-155d-4388-8080-8d3c458aa401" containerID="d6d5fc4017a78a1300fc91b4b4a05b139017e90709fd01256fd8e9e093580969" exitCode=0 Jul 08 19:41:58.917171 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.917131 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerDied","Data":"d6d5fc4017a78a1300fc91b4b4a05b139017e90709fd01256fd8e9e093580969"} Jul 08 19:41:58.920690 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.920666 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" event={"ID":"bfa58b99-1c26-4411-bc9e-25a68ec9e357","Type":"ContainerStarted","Data":"1817aa7e9ac5cba072189b1971d0081948b675401ea797e94a990e99c2562376"} Jul 08 19:41:58.924051 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.924016 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-85ltm" event={"ID":"eb8a42e5-d264-4b5b-b877-0454af519fd4","Type":"ContainerStarted","Data":"c96c18c12aae598c268b7ddd7699a1e1e4feca47519fc5f624a6073fde548a6d"} Jul 08 19:41:58.925961 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.925916 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6phpd" podStartSLOduration=4.972336529 podStartE2EDuration="14.925899849s" totalImagesPullingTime="9.95356332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:58.925163985 +0000 UTC m=+14.582271465" watchObservedRunningTime="2026-07-08 19:41:58.925899849 +0000 UTC m=+14.583007330" Jul 08 19:41:58.928018 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.927993 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ppvcz" event={"ID":"9a05fefd-7dcf-4fdc-833e-6ee404a4de0f","Type":"ContainerStarted","Data":"48a19e0440fc0ffed9f6a614325b4a11c54983805942500308c07f4e2e83f6ad"} Jul 08 19:41:58.940494 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.940442 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dbwfh" podStartSLOduration=4.965325109 podStartE2EDuration="14.94042393s" totalImagesPullingTime="9.975098821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:58.939900907 +0000 UTC m=+14.597008391" watchObservedRunningTime="2026-07-08 19:41:58.94042393 +0000 UTC m=+14.597531411" Jul 08 19:41:58.955572 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:58.955524 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-85ltm" podStartSLOduration=4.023766892 podStartE2EDuration="13.955508891s" totalImagesPullingTime="9.931741999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:58.95502652 +0000 UTC m=+14.612133998" watchObservedRunningTime="2026-07-08 19:41:58.955508891 +0000 UTC m=+14.612616374" Jul 08 19:41:59.015663 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.015495 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ppvcz" podStartSLOduration=4.088537878 podStartE2EDuration="14.015463503s" totalImagesPullingTime="9.926925625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:59.01459768 +0000 UTC m=+14.671705164" watchObservedRunningTime="2026-07-08 19:41:59.015463503 +0000 UTC m=+14.672570984" Jul 08 19:41:59.592946 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.592788 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 19:41:59.808702 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.808562 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:41:59.592944147Z","UUID":"5a29c983-8668-463b-8166-b369ed27c063","Handler":null,"Name":"","Endpoint":""} Jul 08 19:41:59.811348 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.811325 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 19:41:59.811465 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.811367 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 19:41:59.932888 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.932853 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" event={"ID":"5d5b2f68b97376954d14fb66dc3b7019","Type":"ContainerStarted","Data":"67cb92549b355de880b90ed3a46ab922e5fc114ce3e34c821fc0d15401d98349"} Jul 08 19:41:59.935379 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.935268 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" event={"ID":"bfa58b99-1c26-4411-bc9e-25a68ec9e357","Type":"ContainerStarted","Data":"f5406ae6889f738adc5d92e6580faaaa1b664e8431599cef11a48369bb935815"} Jul 08 19:41:59.948780 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:41:59.948729 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-133.ec2.internal" podStartSLOduration=14.948715317 podStartE2EDuration="14.948715317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:59.94856692 +0000 UTC m=+15.605674401" watchObservedRunningTime="2026-07-08 19:41:59.948715317 +0000 UTC m=+15.605822798" Jul 08 19:42:00.852827 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:00.852793 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:00.853003 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:00.852839 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:00.853003 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:00.852793 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:00.853003 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:00.852941 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:00.853175 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:00.853006 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:00.853175 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:00.853065 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:02.463940 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:02.463903 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:02.464624 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.464055 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:02.464624 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.464133 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:18.464113735 +0000 UTC m=+34.121221215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:02.564541 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:02.564499 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:02.564695 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.564672 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:02.564768 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.564743 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:18.564723114 +0000 UTC m=+34.221830589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:02.665216 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:02.665149 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:02.665435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.665345 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:02.665435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.665368 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:02.665435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.665379 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:02.665435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.665426 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:42:18.665412593 +0000 UTC m=+34.322520054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:02.852718 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:02.852639 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:02.852855 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:02.852639 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:02.852855 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.852779 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:02.852855 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:02.852790 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:02.852983 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.852881 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:02.852983 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:02.852965 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:03.285859 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:03.285772 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:42:03.286656 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:03.286638 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:42:04.523231 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:04.523187 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:42:04.523823 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:04.523784 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-85ltm" Jul 08 19:42:04.854009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:04.853925 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:04.854155 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:04.854022 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:04.854155 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:04.854065 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:04.854155 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:04.854093 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:04.854155 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:04.854117 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:04.854379 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:04.854226 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:06.852398 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:06.852362 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:06.852398 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:06.852379 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:06.852968 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:06.852377 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:06.852968 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:06.852482 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:06.852968 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:06.852596 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:06.852968 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:06.852695 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:08.852774 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:08.852734 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:08.853435 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:08.852734 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:08.853435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:08.852892 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:08.853435 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:08.852743 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:08.853435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:08.852946 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:08.853435 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:08.853021 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:10.853366 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.853065 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:10.853711 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.853065 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:10.853711 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:10.853446 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:10.853711 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:10.853540 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:10.853711 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.853092 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:10.853711 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:10.853643 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:10.960042 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.959949 2569 generic.go:356] "Generic (PLEG): container finished" podID="9ccc8460-155d-4388-8080-8d3c458aa401" containerID="a3a1989912400d287f304b5187da20e99df8cae82bac10961e5724b066bbb246" exitCode=0 Jul 08 19:42:10.960042 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.960009 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerDied","Data":"a3a1989912400d287f304b5187da20e99df8cae82bac10961e5724b066bbb246"} Jul 08 19:42:10.962262 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.962238 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" event={"ID":"bfa58b99-1c26-4411-bc9e-25a68ec9e357","Type":"ContainerStarted","Data":"92f63abb93e665df93fdd6d4c78a87ef4ff32446b02e5719b288d2674a213a86"} Jul 08 19:42:10.965411 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.965384 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"066b60d57e43185d8ee2dde5691004cf16b4c28918ad17e48a7a6cd7372575a4"} Jul 08 19:42:10.965522 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.965456 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"f5dcc35b76336cd55b7558eba349204df5d6c990dbc62407ce330d41a89ed1c6"} Jul 08 19:42:10.965522 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.965492 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"c61f7f9f8f79ddf47ed771a54be4e226266ebd2c56990b907aead638dc604061"} Jul 08 19:42:10.965627 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.965523 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"fe2ab8ad28bf42432e74b6bd6a673d78cb35b6146c71a55f2c1a8c0684174eed"} Jul 08 19:42:10.965627 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.965567 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"ee71b2d0b5db331505a1bc9f5705186e6a993ceb98a954509af0d231efde92c5"} Jul 08 19:42:10.965627 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.965601 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"7f8b2f4fcec3654826f213868b30a92d877b5e6d3f42e4022226495d88079360"} Jul 08 19:42:10.966750 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:10.966728 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x6qqv" event={"ID":"9444ef70-a6fa-4055-9643-60bff9cac87b","Type":"ContainerStarted","Data":"6e63beccd0a1eec925804f947d83cbbb0479331cd968f3f6583d4fb26be52b2e"} Jul 08 19:42:11.016505 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:11.016457 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btxz6" podStartSLOduration=4.427334289 podStartE2EDuration="26.016441948s" totalImagesPullingTime="21.589107659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:11.01642633 +0000 UTC m=+26.673533818" watchObservedRunningTime="2026-07-08 19:42:11.016441948 +0000 UTC m=+26.673549428" Jul 08 19:42:11.016859 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:11.016840 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x6qqv" podStartSLOduration=3.122096843 podStartE2EDuration="26.016834586s" totalImagesPullingTime="22.894737743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:11.0008011 +0000 UTC m=+26.657908599" watchObservedRunningTime="2026-07-08 19:42:11.016834586 +0000 UTC m=+26.673942066" Jul 08 19:42:12.852239 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:12.852216 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:12.852666 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:12.852216 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:12.852666 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:12.852303 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:12.852666 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:12.852342 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:12.852666 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:12.852407 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:12.852666 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:12.852464 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:12.972327 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:12.972296 2569 generic.go:356] "Generic (PLEG): container finished" podID="9ccc8460-155d-4388-8080-8d3c458aa401" containerID="bb7e2034f900c9717939a98859390cec5a4e36d368b29bca1402a963d63887a5" exitCode=0 Jul 08 19:42:12.972490 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:12.972374 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerDied","Data":"bb7e2034f900c9717939a98859390cec5a4e36d368b29bca1402a963d63887a5"} Jul 08 19:42:12.976458 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:12.976420 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"1e46562e156b4ecae9772e2a392d20f0969f8b6a97ae3aad76cea79d4f80cf8e"} Jul 08 19:42:14.855360 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:14.854510 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:14.855809 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:14.855546 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:14.855873 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:14.855515 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:14.855923 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:14.855872 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:14.855961 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:14.855945 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:14.856018 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:14.856003 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:14.983601 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:14.983577 2569 generic.go:356] "Generic (PLEG): container finished" podID="9ccc8460-155d-4388-8080-8d3c458aa401" containerID="89982108d34396c2fc010e92ac40a8adda79828e0bc8a5afd00a6767fa41abc8" exitCode=0 Jul 08 19:42:14.983693 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:14.983635 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerDied","Data":"89982108d34396c2fc010e92ac40a8adda79828e0bc8a5afd00a6767fa41abc8"} Jul 08 19:42:15.990609 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:15.990361 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" event={"ID":"39a4d2a3-681f-4b52-ba42-7f32b11189bb","Type":"ContainerStarted","Data":"75078392d1db009c2b6659f0d9798e097277e8561dc405db4ed9dd3837002640"} Jul 08 19:42:15.991336 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:15.990675 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:42:15.991336 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:15.990721 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:42:15.991336 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:15.990757 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:42:16.006956 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.006929 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:42:16.007177 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.007151 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:42:16.017649 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.017579 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" podStartSLOduration=8.126731429 podStartE2EDuration="31.017559209s" totalImagesPullingTime="22.89082778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:16.015659777 +0000 UTC m=+31.672767293" watchObservedRunningTime="2026-07-08 19:42:16.017559209 +0000 UTC m=+31.674666691" Jul 08 19:42:16.856066 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.856039 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:16.856245 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.856150 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:16.856245 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:16.856152 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:16.856379 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:16.856272 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:16.856379 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.856157 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:16.856467 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:16.856380 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:16.940174 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.940143 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rkvt6"] Jul 08 19:42:16.942811 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.942749 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rdnjw"] Jul 08 19:42:16.943416 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.943394 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvcrj"] Jul 08 19:42:16.992784 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.992755 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:16.992784 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.992755 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:16.993235 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:16.992755 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:16.993235 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:16.992863 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:16.993235 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:16.992940 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:16.993235 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:16.992975 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:18.476419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:18.476385 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:18.476943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.476524 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:18.476943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.476585 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:50.476569283 +0000 UTC m=+66.133676741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:42:18.577387 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:18.577343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:18.577573 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.577518 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:18.577639 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.577604 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret podName:be2ea6fc-bd9b-4078-9634-1c31078e4661 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:50.577583906 +0000 UTC m=+66.234691381 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret") pod "global-pull-secret-syncer-rdnjw" (UID: "be2ea6fc-bd9b-4078-9634-1c31078e4661") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:42:18.677962 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:18.677930 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:18.678122 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.678059 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:42:18.678122 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.678075 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:42:18.678122 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.678083 2569 projected.go:196] Error preparing data for projected volume kube-api-access-lkhrg for pod openshift-network-diagnostics/network-check-target-rkvt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:18.678267 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.678128 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg podName:0a00dc7d-edcd-4484-949c-564b1711f34e nodeName:}" failed. No retries permitted until 2026-07-08 19:42:50.678117374 +0000 UTC m=+66.335224831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhrg" (UniqueName: "kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg") pod "network-check-target-rkvt6" (UID: "0a00dc7d-edcd-4484-949c-564b1711f34e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:42:18.856263 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:18.856165 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:18.856418 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:18.856308 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:18.856418 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.856308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:18.856418 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:18.856334 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:18.856536 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.856419 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:18.856536 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:18.856508 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:20.856849 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:20.856819 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:20.857395 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:20.856818 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:20.857395 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:20.856928 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:20.857395 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:20.857019 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:20.857395 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:20.856824 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:20.857395 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:20.857124 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:22.008253 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:22.007992 2569 generic.go:356] "Generic (PLEG): container finished" podID="9ccc8460-155d-4388-8080-8d3c458aa401" containerID="f3af4aaee89c1cb7fc3a765bc41332c4885659d2fd956bec49ff5481e23f3f87" exitCode=0 Jul 08 19:42:22.008253 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:22.008073 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerDied","Data":"f3af4aaee89c1cb7fc3a765bc41332c4885659d2fd956bec49ff5481e23f3f87"} Jul 08 19:42:22.852934 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:22.852846 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:22.852934 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:22.852874 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:22.853146 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:22.852852 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:22.853146 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:22.852966 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rkvt6" podUID="0a00dc7d-edcd-4484-949c-564b1711f34e" Jul 08 19:42:22.853146 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:22.853066 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:42:22.853377 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:22.853158 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rdnjw" podUID="be2ea6fc-bd9b-4078-9634-1c31078e4661" Jul 08 19:42:23.013428 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.013392 2569 generic.go:356] "Generic (PLEG): container finished" podID="9ccc8460-155d-4388-8080-8d3c458aa401" containerID="3eb09bec4bb03ea92949e49b7f1728f824f7a1a116fb5b5d2cfb781a524c934f" exitCode=0 Jul 08 19:42:23.013792 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.013447 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerDied","Data":"3eb09bec4bb03ea92949e49b7f1728f824f7a1a116fb5b5d2cfb781a524c934f"} Jul 08 19:42:23.107786 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.107741 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-133.ec2.internal" event="NodeReady" Jul 08 19:42:23.107903 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.107843 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 19:42:23.145870 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.145843 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59586d4b44-92bwp"] Jul 08 19:42:23.159943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.159919 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm"] Jul 08 19:42:23.160054 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.160038 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.163480 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.163396 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 19:42:23.164965 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.164705 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 19:42:23.164965 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.164741 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 19:42:23.164965 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.164802 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4fdwr\"" Jul 08 19:42:23.170815 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.170350 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r88l4"] Jul 08 19:42:23.170815 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.170444 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.170983 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.170897 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 19:42:23.173497 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.173479 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z9rcf\"" Jul 08 19:42:23.173579 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.173508 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 19:42:23.173579 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.173480 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 19:42:23.186256 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.186237 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm"] Jul 08 19:42:23.186348 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.186266 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59586d4b44-92bwp"] Jul 08 19:42:23.186348 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.186275 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.186348 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.186282 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-24rzx"] Jul 08 19:42:23.189055 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.189036 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 19:42:23.189152 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.189060 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 19:42:23.189152 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.189088 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w5s5j\"" Jul 08 19:42:23.202809 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.202789 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r88l4"] Jul 08 19:42:23.202897 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.202820 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-24rzx"] Jul 08 19:42:23.202935 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.202901 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.205729 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.205711 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4qkct\"" Jul 08 19:42:23.205821 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.205742 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 19:42:23.205821 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.205795 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 19:42:23.205902 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.205796 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 19:42:23.308270 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308443 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308283 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-installation-pull-secrets\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308443 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308335 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa3e3326-af03-43a2-a7d0-b91d79924d6c-ca-trust-extracted\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308443 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308384 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt6h6\" (UniqueName: \"kubernetes.io/projected/c24c75a7-a8f5-4534-85e6-8f9431128780-kube-api-access-rt6h6\") pod \"ingress-canary-24rzx\" (UID: \"c24c75a7-a8f5-4534-85e6-8f9431128780\") " pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.308443 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308418 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-certificates\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308443 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308434 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.308672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308460 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-trusted-ca\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308500 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c24c75a7-a8f5-4534-85e6-8f9431128780-cert\") pod \"ingress-canary-24rzx\" (UID: \"c24c75a7-a8f5-4534-85e6-8f9431128780\") " pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.308672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308535 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dglz\" (UniqueName: \"kubernetes.io/projected/ec911e95-6f22-4710-9bf4-d3ebf667c684-kube-api-access-2dglz\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.308672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308593 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.308672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308619 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec911e95-6f22-4710-9bf4-d3ebf667c684-tmp-dir\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.308672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308669 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-image-registry-private-configuration\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308883 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-bound-sa-token\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308883 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308726 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec911e95-6f22-4710-9bf4-d3ebf667c684-config-volume\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.308883 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308762 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htqbr\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-kube-api-access-htqbr\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.308883 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.308782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311603ab-67b8-4d51-92b9-2522a8e68cb3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.409495 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409426 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.409495 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409468 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-installation-pull-secrets\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.409495 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409487 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa3e3326-af03-43a2-a7d0-b91d79924d6c-ca-trust-extracted\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409512 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt6h6\" (UniqueName: \"kubernetes.io/projected/c24c75a7-a8f5-4534-85e6-8f9431128780-kube-api-access-rt6h6\") pod \"ingress-canary-24rzx\" (UID: \"c24c75a7-a8f5-4534-85e6-8f9431128780\") " pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409564 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-certificates\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409599 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.409602 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.409619 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-trusted-ca\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409638 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c24c75a7-a8f5-4534-85e6-8f9431128780-cert\") pod \"ingress-canary-24rzx\" (UID: \"c24c75a7-a8f5-4534-85e6-8f9431128780\") " pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.409676 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:42:23.909654474 +0000 UTC m=+39.566761946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409697 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dglz\" (UniqueName: \"kubernetes.io/projected/ec911e95-6f22-4710-9bf4-d3ebf667c684-kube-api-access-2dglz\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409731 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409759 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec911e95-6f22-4710-9bf4-d3ebf667c684-tmp-dir\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.409776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409789 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-image-registry-private-configuration\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.410358 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409816 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-bound-sa-token\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.410358 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409841 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec911e95-6f22-4710-9bf4-d3ebf667c684-config-volume\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.410358 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409880 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqbr\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-kube-api-access-htqbr\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.410358 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.409909 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311603ab-67b8-4d51-92b9-2522a8e68cb3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.410358 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.410143 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa3e3326-af03-43a2-a7d0-b91d79924d6c-ca-trust-extracted\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.410358 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.410326 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-certificates\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.410609 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.410421 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:23.410609 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.410476 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:23.910457739 +0000 UTC m=+39.567565204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:23.410703 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.410675 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-trusted-ca\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.410760 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.410677 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311603ab-67b8-4d51-92b9-2522a8e68cb3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.410760 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.410724 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec911e95-6f22-4710-9bf4-d3ebf667c684-tmp-dir\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.410847 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.410773 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:23.410847 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.410776 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec911e95-6f22-4710-9bf4-d3ebf667c684-config-volume\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.410847 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.410841 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:23.910824902 +0000 UTC m=+39.567932360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:23.413990 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.413970 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c24c75a7-a8f5-4534-85e6-8f9431128780-cert\") pod \"ingress-canary-24rzx\" (UID: \"c24c75a7-a8f5-4534-85e6-8f9431128780\") " pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.413990 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.413979 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-installation-pull-secrets\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.413990 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.413979 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-image-registry-private-configuration\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.418712 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.418690 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dglz\" (UniqueName: \"kubernetes.io/projected/ec911e95-6f22-4710-9bf4-d3ebf667c684-kube-api-access-2dglz\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.418958 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.418925 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-bound-sa-token\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.419057 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.419007 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt6h6\" (UniqueName: \"kubernetes.io/projected/c24c75a7-a8f5-4534-85e6-8f9431128780-kube-api-access-rt6h6\") pod \"ingress-canary-24rzx\" (UID: \"c24c75a7-a8f5-4534-85e6-8f9431128780\") " pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.419639 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.419623 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqbr\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-kube-api-access-htqbr\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.510624 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.510591 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-24rzx" Jul 08 19:42:23.710782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.710567 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-24rzx"] Jul 08 19:42:23.714054 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:42:23.714030 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24c75a7_a8f5_4534_85e6_8f9431128780.slice/crio-5c485928cb75e44176be430d55ab6cfe8082a50cd70ac7754fae45cf0d2dd41b WatchSource:0}: Error finding container 5c485928cb75e44176be430d55ab6cfe8082a50cd70ac7754fae45cf0d2dd41b: Status 404 returned error can't find the container with id 5c485928cb75e44176be430d55ab6cfe8082a50cd70ac7754fae45cf0d2dd41b Jul 08 19:42:23.914040 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.914003 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:23.914247 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.914078 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:23.914247 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:23.914103 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:23.914247 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914179 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:23.914247 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914189 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:23.914247 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914222 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:23.914495 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914189 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:23.914495 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914269 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:24.914254446 +0000 UTC m=+40.571361904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:23.914495 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914286 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:42:24.914279638 +0000 UTC m=+40.571387096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:23.914495 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:23.914296 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:24.914290846 +0000 UTC m=+40.571398304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:24.019473 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.019373 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" event={"ID":"9ccc8460-155d-4388-8080-8d3c458aa401","Type":"ContainerStarted","Data":"f2d3cc05bdd60980f462b1782ba4abb2077cdb66e85e5eada4f4f5cab367e786"} Jul 08 19:42:24.020432 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.020410 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-24rzx" event={"ID":"c24c75a7-a8f5-4534-85e6-8f9431128780","Type":"ContainerStarted","Data":"5c485928cb75e44176be430d55ab6cfe8082a50cd70ac7754fae45cf0d2dd41b"} Jul 08 19:42:24.044016 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.043971 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jdhtx" podStartSLOduration=9.279771978 podStartE2EDuration="39.04395735s" totalImagesPullingTime="29.764185372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:24.041886195 +0000 UTC m=+39.698993674" watchObservedRunningTime="2026-07-08 19:42:24.04395735 +0000 UTC m=+39.701064830" Jul 08 19:42:24.856218 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.856176 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:24.856397 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.856176 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:24.856474 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.856243 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:24.859969 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.859940 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:42:24.860073 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.859970 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:42:24.861466 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.860926 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:42:24.861466 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.861056 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:42:24.861466 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.861316 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dpn4j\"" Jul 08 19:42:24.861466 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.861386 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84p6s\"" Jul 08 19:42:24.922847 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.922818 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:24.923015 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.922909 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:24.923015 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:24.922947 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:24.923015 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.922995 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:24.923015 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.923015 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:24.923223 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.923043 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:24.923223 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.923074 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:24.923223 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.923085 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:42:26.923064541 +0000 UTC m=+42.580172014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:24.923223 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.923098 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:26.923092539 +0000 UTC m=+42.580199997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:24.923223 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:24.923136 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:26.923120857 +0000 UTC m=+42.580228330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:26.939184 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:26.939150 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:26.939246 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:26.939271 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939298 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939316 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939369 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:42:30.939354193 +0000 UTC m=+46.596461651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939369 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939386 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939412 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:30.939401059 +0000 UTC m=+46.596508661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:26.939608 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:26.939433 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:30.939417929 +0000 UTC m=+46.596525410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:27.029333 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:27.029295 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-24rzx" event={"ID":"c24c75a7-a8f5-4534-85e6-8f9431128780","Type":"ContainerStarted","Data":"59339d8e6cd7f57f4de2a4cc2324e0af88d4631cc84f45f3e1e4c65f0254e13d"} Jul 08 19:42:27.045383 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:27.045340 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-24rzx" podStartSLOduration=0.932212107 podStartE2EDuration="4.045325282s" totalImagesPullingTime="3.113113175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:27.045052675 +0000 UTC m=+42.702160168" watchObservedRunningTime="2026-07-08 19:42:27.045325282 +0000 UTC m=+42.702432762" Jul 08 19:42:29.049037 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.049009 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t"] Jul 08 19:42:29.065279 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.065255 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t"] Jul 08 19:42:29.065407 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.065366 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.069880 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.069474 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-f8x7p\"" Jul 08 19:42:29.069880 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.069829 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 19:42:29.070141 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.070123 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 19:42:29.070141 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.070130 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 19:42:29.070301 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.069827 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc"] Jul 08 19:42:29.070441 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.070366 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 19:42:29.083661 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.083631 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg"] Jul 08 19:42:29.083806 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.083782 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.086816 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.086790 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 19:42:29.086943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.086857 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 19:42:29.086943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.086905 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 19:42:29.086943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.086856 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 19:42:29.101827 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.101805 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc"] Jul 08 19:42:29.101941 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.101838 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg"] Jul 08 19:42:29.101941 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.101913 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.104765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.104744 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 19:42:29.156696 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.156655 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-674b855b74-xtx4t\" (UID: \"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.156866 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.156815 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmvkr\" (UniqueName: \"kubernetes.io/projected/4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd-kube-api-access-dmvkr\") pod \"managed-serviceaccount-addon-agent-674b855b74-xtx4t\" (UID: \"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.257525 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257490 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.257675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257536 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.257675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257611 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmvkr\" (UniqueName: \"kubernetes.io/projected/4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd-kube-api-access-dmvkr\") pod \"managed-serviceaccount-addon-agent-674b855b74-xtx4t\" (UID: \"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.257675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-tmp\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.257772 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257684 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbsst\" (UniqueName: \"kubernetes.io/projected/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-kube-api-access-nbsst\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.257772 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257705 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trv6g\" (UniqueName: \"kubernetes.io/projected/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-kube-api-access-trv6g\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.257772 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257737 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-hub\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.257958 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257793 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.257958 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257860 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.257958 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257887 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-ca\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.257958 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.257935 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-674b855b74-xtx4t\" (UID: \"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.262006 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.261895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-674b855b74-xtx4t\" (UID: \"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.266922 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.266893 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmvkr\" (UniqueName: \"kubernetes.io/projected/4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd-kube-api-access-dmvkr\") pod \"managed-serviceaccount-addon-agent-674b855b74-xtx4t\" (UID: \"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.358921 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.358881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.358921 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.358929 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.359118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.358967 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-tmp\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.359118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.358992 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbsst\" (UniqueName: \"kubernetes.io/projected/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-kube-api-access-nbsst\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.359118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359008 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trv6g\" (UniqueName: \"kubernetes.io/projected/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-kube-api-access-trv6g\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.359118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-hub\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.359118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359047 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.359361 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359240 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.359361 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359272 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-ca\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.359495 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359472 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-tmp\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.359875 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.359850 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.361718 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.361690 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.361815 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.361743 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-hub\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.361815 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.361764 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.362052 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.362032 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-ca\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.362111 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.362048 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.367687 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.367666 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbsst\" (UniqueName: \"kubernetes.io/projected/7e9004e7-9f85-408f-b1ca-f2deaa0f82f7-kube-api-access-nbsst\") pod \"klusterlet-addon-workmgr-5d747fbc46-fs9vg\" (UID: \"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.367687 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.367678 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trv6g\" (UniqueName: \"kubernetes.io/projected/15f9d4fe-12ca-4421-a7ff-f6128ee3b099-kube-api-access-trv6g\") pod \"cluster-proxy-proxy-agent-67fb889cd5-gtqgc\" (UID: \"15f9d4fe-12ca-4421-a7ff-f6128ee3b099\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.386963 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.386936 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:42:29.394118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.394099 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:42:29.423447 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.423420 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:29.550724 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.550294 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t"] Jul 08 19:42:29.552967 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:42:29.552929 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b47cdc1_aa7f_48c9_a301_af2e9b34ebdd.slice/crio-a4f1e3741a4013843b4fdc6a46c78555bb89e3f566c49998e0647ad9cdb61e1f WatchSource:0}: Error finding container a4f1e3741a4013843b4fdc6a46c78555bb89e3f566c49998e0647ad9cdb61e1f: Status 404 returned error can't find the container with id a4f1e3741a4013843b4fdc6a46c78555bb89e3f566c49998e0647ad9cdb61e1f Jul 08 19:42:29.568938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.568909 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc"] Jul 08 19:42:29.572893 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:42:29.572848 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15f9d4fe_12ca_4421_a7ff_f6128ee3b099.slice/crio-c52075d9bca191e20323948dc6adc2eee9cf428c6eae7e9f024c1faa1921ee41 WatchSource:0}: Error finding container c52075d9bca191e20323948dc6adc2eee9cf428c6eae7e9f024c1faa1921ee41: Status 404 returned error can't find the container with id c52075d9bca191e20323948dc6adc2eee9cf428c6eae7e9f024c1faa1921ee41 Jul 08 19:42:29.590329 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:29.590304 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg"] Jul 08 19:42:29.593423 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:42:29.593404 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e9004e7_9f85_408f_b1ca_f2deaa0f82f7.slice/crio-8cbdf3b58cbe94d9f68e9aa0525e891d918996d807c9f6d8a6c7652298a18d52 WatchSource:0}: Error finding container 8cbdf3b58cbe94d9f68e9aa0525e891d918996d807c9f6d8a6c7652298a18d52: Status 404 returned error can't find the container with id 8cbdf3b58cbe94d9f68e9aa0525e891d918996d807c9f6d8a6c7652298a18d52 Jul 08 19:42:30.037882 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:30.037848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" event={"ID":"15f9d4fe-12ca-4421-a7ff-f6128ee3b099","Type":"ContainerStarted","Data":"c52075d9bca191e20323948dc6adc2eee9cf428c6eae7e9f024c1faa1921ee41"} Jul 08 19:42:30.038982 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:30.038958 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" event={"ID":"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd","Type":"ContainerStarted","Data":"a4f1e3741a4013843b4fdc6a46c78555bb89e3f566c49998e0647ad9cdb61e1f"} Jul 08 19:42:30.039952 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:30.039933 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" event={"ID":"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7","Type":"ContainerStarted","Data":"8cbdf3b58cbe94d9f68e9aa0525e891d918996d807c9f6d8a6c7652298a18d52"} Jul 08 19:42:30.974994 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:30.974955 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:30.975798 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:30.975049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:30.975798 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:30.975085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:30.975798 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.975231 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:30.975798 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.975245 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:30.975798 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.975298 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:42:38.975281721 +0000 UTC m=+54.632389179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:30.976042 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.975965 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:30.976042 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.976021 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:38.976004905 +0000 UTC m=+54.633112364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:30.976150 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.976082 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:30.976150 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:30.976116 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:38.97610443 +0000 UTC m=+54.633211888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:35.062026 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:35.061980 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" event={"ID":"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7","Type":"ContainerStarted","Data":"2d09c101d3b6528b0c723be2ca8c1f4441d75540990b57e92e1848d189e1db5d"} Jul 08 19:42:35.062519 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:35.062319 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:35.063763 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:35.063710 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" event={"ID":"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd","Type":"ContainerStarted","Data":"f141fcddcd845e64d5024360a384936365c96f707ee335cb68bf1e38d3bb44e6"} Jul 08 19:42:35.064145 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:35.064122 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:42:35.078384 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:35.078338 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" podStartSLOduration=0.805203416 podStartE2EDuration="6.078322391s" totalImagesPullingTime="5.273118975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:35.077269164 +0000 UTC m=+50.734376645" watchObservedRunningTime="2026-07-08 19:42:35.078322391 +0000 UTC m=+50.735429870" Jul 08 19:42:35.111797 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:35.111668 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" podStartSLOduration=0.815544788 podStartE2EDuration="6.111650916s" totalImagesPullingTime="5.296106128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:35.110376927 +0000 UTC m=+50.767484422" watchObservedRunningTime="2026-07-08 19:42:35.111650916 +0000 UTC m=+50.768758396" Jul 08 19:42:38.074106 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:38.074071 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" event={"ID":"15f9d4fe-12ca-4421-a7ff-f6128ee3b099","Type":"ContainerStarted","Data":"46092b337ff86ea8ab580fc32026c46254c354736b31366d4a3fcddd7ee473ff"} Jul 08 19:42:38.074494 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:38.074121 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" event={"ID":"15f9d4fe-12ca-4421-a7ff-f6128ee3b099","Type":"ContainerStarted","Data":"db27d34ad5cd72a5f2b9b3396a84efeff9397e8f0370f24a466dcbc87a0e4d29"} Jul 08 19:42:38.074494 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:38.074142 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" event={"ID":"15f9d4fe-12ca-4421-a7ff-f6128ee3b099","Type":"ContainerStarted","Data":"d336fab02fbd63bd06570a7bbc9858270a3d8e240042a7c0753bbf216b1ed1c4"} Jul 08 19:42:38.095872 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:38.095828 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" podStartSLOduration=1.593464906 podStartE2EDuration="9.095813624s" totalImagesPullingTime="7.502348718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:38.094287847 +0000 UTC m=+53.751395326" watchObservedRunningTime="2026-07-08 19:42:38.095813624 +0000 UTC m=+53.752921103" Jul 08 19:42:39.044684 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:39.044639 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:39.044705 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:39.044763 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044802 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044822 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044853 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044883 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:42:55.044867308 +0000 UTC m=+70.701974765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044908 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:55.044895602 +0000 UTC m=+70.702003066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044853 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:39.044943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:39.044942 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:42:55.044931566 +0000 UTC m=+70.702039027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:48.005903 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:48.005875 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92tws" Jul 08 19:42:50.536054 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.536007 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:42:50.539278 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.539255 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:42:50.546680 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:50.546657 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:42:50.546777 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:50.546750 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:43:54.546726487 +0000 UTC m=+130.203833963 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : secret "metrics-daemon-secret" not found Jul 08 19:42:50.637404 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.637363 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:50.640942 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.640921 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:42:50.650821 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.650796 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be2ea6fc-bd9b-4078-9634-1c31078e4661-original-pull-secret\") pod \"global-pull-secret-syncer-rdnjw\" (UID: \"be2ea6fc-bd9b-4078-9634-1c31078e4661\") " pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:50.685220 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.685169 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rdnjw" Jul 08 19:42:50.738491 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.738461 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:50.741765 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.741741 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:42:50.751724 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.751689 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:42:50.763038 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.763003 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkhrg\" (UniqueName: \"kubernetes.io/projected/0a00dc7d-edcd-4484-949c-564b1711f34e-kube-api-access-lkhrg\") pod \"network-check-target-rkvt6\" (UID: \"0a00dc7d-edcd-4484-949c-564b1711f34e\") " pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:50.806228 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.806182 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rdnjw"] Jul 08 19:42:50.809194 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:42:50.809169 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2ea6fc_bd9b_4078_9634_1c31078e4661.slice/crio-daa8e40e721f3b0b9483e9a6abc75ec8553550894cbed96bdb9963670c18fc3e WatchSource:0}: Error finding container daa8e40e721f3b0b9483e9a6abc75ec8553550894cbed96bdb9963670c18fc3e: Status 404 returned error can't find the container with id daa8e40e721f3b0b9483e9a6abc75ec8553550894cbed96bdb9963670c18fc3e Jul 08 19:42:50.974123 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.974095 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84p6s\"" Jul 08 19:42:50.981482 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:50.981464 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:51.100528 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:51.100441 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rkvt6"] Jul 08 19:42:51.104048 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:42:51.104022 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a00dc7d_edcd_4484_949c_564b1711f34e.slice/crio-51268997d83f6f566a12f908898380709ff3e660fa4546c2c279d4676af196e4 WatchSource:0}: Error finding container 51268997d83f6f566a12f908898380709ff3e660fa4546c2c279d4676af196e4: Status 404 returned error can't find the container with id 51268997d83f6f566a12f908898380709ff3e660fa4546c2c279d4676af196e4 Jul 08 19:42:51.116031 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:51.115995 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rkvt6" event={"ID":"0a00dc7d-edcd-4484-949c-564b1711f34e","Type":"ContainerStarted","Data":"51268997d83f6f566a12f908898380709ff3e660fa4546c2c279d4676af196e4"} Jul 08 19:42:51.117131 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:51.117108 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rdnjw" event={"ID":"be2ea6fc-bd9b-4078-9634-1c31078e4661","Type":"ContainerStarted","Data":"daa8e40e721f3b0b9483e9a6abc75ec8553550894cbed96bdb9963670c18fc3e"} Jul 08 19:42:55.074714 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:55.074678 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:55.074736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:55.074793 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074856 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074884 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074861 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074943 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074931 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:43:27.074910496 +0000 UTC m=+102.732017967 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074987 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:43:27.074975065 +0000 UTC m=+102.732082523 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:42:55.075170 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:42:55.074997 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:43:27.074992159 +0000 UTC m=+102.732099617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:42:55.133339 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:55.133302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rkvt6" event={"ID":"0a00dc7d-edcd-4484-949c-564b1711f34e","Type":"ContainerStarted","Data":"ca0865311b21b76953d47090c49762dfe2e42f5ca84fd4bad891c99b657cd9f5"} Jul 08 19:42:55.133511 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:55.133444 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:42:55.150668 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:55.150620 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rkvt6" podStartSLOduration=66.670471162 podStartE2EDuration="1m10.15060493s" totalImagesPullingTime="3.480133768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:55.149376271 +0000 UTC m=+70.806483765" watchObservedRunningTime="2026-07-08 19:42:55.15060493 +0000 UTC m=+70.807712409" Jul 08 19:42:57.141899 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:57.141865 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rdnjw" event={"ID":"be2ea6fc-bd9b-4078-9634-1c31078e4661","Type":"ContainerStarted","Data":"d90225969937441958699d3cd0f8f1f38c64402c0f9f7c57a6a1b66d8a3e1fa5"} Jul 08 19:42:57.157741 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:42:57.157691 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rdnjw" podStartSLOduration=66.611723367 podStartE2EDuration="1m12.157673646s" totalImagesPullingTime="5.545950279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:42:57.157607598 +0000 UTC m=+72.814715079" watchObservedRunningTime="2026-07-08 19:42:57.157673646 +0000 UTC m=+72.814781127" Jul 08 19:43:26.140227 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:43:26.140172 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rkvt6" Jul 08 19:43:27.115420 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:43:27.115383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:43:27.115594 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:43:27.115430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:43:27.115594 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:43:27.115463 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:43:27.115594 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115535 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:43:27.115594 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115550 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:43:27.115594 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115588 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:43:27.115758 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115602 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:43:27.115758 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115610 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:44:31.11559173 +0000 UTC m=+166.772699191 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:43:27.115758 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115626 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:44:31.11561992 +0000 UTC m=+166.772727378 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:43:27.115758 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:27.115637 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:44:31.115632094 +0000 UTC m=+166.772739552 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:43:54.619154 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:43:54.619113 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:43:54.619645 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:54.619301 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:43:54.619645 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:43:54.619369 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:45:56.619352785 +0000 UTC m=+252.276460242 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : secret "metrics-daemon-secret" not found Jul 08 19:44:26.172130 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:26.172080 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" Jul 08 19:44:26.179231 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:26.179184 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" podUID="311603ab-67b8-4d51-92b9-2522a8e68cb3" Jul 08 19:44:26.197542 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:26.197509 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-r88l4" podUID="ec911e95-6f22-4710-9bf4-d3ebf667c684" Jul 08 19:44:26.415392 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:26.415363 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:44:26.415583 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:26.415363 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r88l4" Jul 08 19:44:26.415583 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:26.415369 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:44:27.879059 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:27.879015 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:44:31.203907 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:31.203868 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:31.203919 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:31.203951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204015 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204044 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204080 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls podName:ec911e95-6f22-4710-9bf4-d3ebf667c684 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:33.204063413 +0000 UTC m=+288.861170871 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls") pod "dns-default-r88l4" (UID: "ec911e95-6f22-4710-9bf4-d3ebf667c684") : secret "dns-default-metrics-tls" not found Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204089 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204108 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59586d4b44-92bwp: secret "image-registry-tls" not found Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204094 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert podName:311603ab-67b8-4d51-92b9-2522a8e68cb3 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:33.204087844 +0000 UTC m=+288.861195302 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-bfbrm" (UID: "311603ab-67b8-4d51-92b9-2522a8e68cb3") : secret "networking-console-plugin-cert" not found Jul 08 19:44:31.204368 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:44:31.204175 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls podName:fa3e3326-af03-43a2-a7d0-b91d79924d6c nodeName:}" failed. No retries permitted until 2026-07-08 19:46:33.204162434 +0000 UTC m=+288.861269892 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls") pod "image-registry-59586d4b44-92bwp" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c") : secret "image-registry-tls" not found Jul 08 19:44:35.063364 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.063320 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" podUID="7e9004e7-9f85-408f-b1ca-f2deaa0f82f7" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.12:8000/readyz\": dial tcp 10.132.0.12:8000: connect: connection refused" Jul 08 19:44:35.442910 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.442877 2569 generic.go:356] "Generic (PLEG): container finished" podID="4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd" containerID="f141fcddcd845e64d5024360a384936365c96f707ee335cb68bf1e38d3bb44e6" exitCode=255 Jul 08 19:44:35.443082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.442947 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" event={"ID":"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd","Type":"ContainerDied","Data":"f141fcddcd845e64d5024360a384936365c96f707ee335cb68bf1e38d3bb44e6"} Jul 08 19:44:35.443310 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.443292 2569 scope.go:117] "RemoveContainer" containerID="f141fcddcd845e64d5024360a384936365c96f707ee335cb68bf1e38d3bb44e6" Jul 08 19:44:35.444517 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.444492 2569 generic.go:356] "Generic (PLEG): container finished" podID="7e9004e7-9f85-408f-b1ca-f2deaa0f82f7" containerID="2d09c101d3b6528b0c723be2ca8c1f4441d75540990b57e92e1848d189e1db5d" exitCode=1 Jul 08 19:44:35.444631 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.444525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" event={"ID":"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7","Type":"ContainerDied","Data":"2d09c101d3b6528b0c723be2ca8c1f4441d75540990b57e92e1848d189e1db5d"} Jul 08 19:44:35.444805 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:35.444791 2569 scope.go:117] "RemoveContainer" containerID="2d09c101d3b6528b0c723be2ca8c1f4441d75540990b57e92e1848d189e1db5d" Jul 08 19:44:36.449089 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:36.449043 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" event={"ID":"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7","Type":"ContainerStarted","Data":"9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c"} Jul 08 19:44:36.449654 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:36.449417 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:44:36.450017 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:36.449992 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:44:36.450751 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:36.450729 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" event={"ID":"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd","Type":"ContainerStarted","Data":"dfcf4527528dbce1d7dba2054cc661a31bdc5d34da602c7d1f69369cf7d485dc"} Jul 08 19:44:42.852093 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:44:42.851995 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:45:56.660645 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:45:56.660606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:45:56.661146 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:45:56.660724 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:45:56.661146 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:45:56.660785 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs podName:42b16c47-efdc-4d44-9501-99711b3e6487 nodeName:}" failed. No retries permitted until 2026-07-08 19:47:58.66077151 +0000 UTC m=+374.317878968 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs") pod "network-metrics-daemon-dvcrj" (UID: "42b16c47-efdc-4d44-9501-99711b3e6487") : secret "metrics-daemon-secret" not found Jul 08 19:46:10.683377 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.683291 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sgkx5"] Jul 08 19:46:10.686332 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.686314 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.691093 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.691065 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:46:10.691241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.691076 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:46:10.691241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.691102 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:46:10.691455 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.691442 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:46:10.692449 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.692432 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8lf2\"" Jul 08 19:46:10.702850 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.702826 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sgkx5"] Jul 08 19:46:10.752431 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.752395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aea20af8-ca7e-4d2f-af08-6ec4287357ad-data-volume\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.752431 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.752432 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aea20af8-ca7e-4d2f-af08-6ec4287357ad-crio-socket\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.752675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.752463 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87sv\" (UniqueName: \"kubernetes.io/projected/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-api-access-x87sv\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.752675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.752506 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.752675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.752578 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.852939 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.852902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aea20af8-ca7e-4d2f-af08-6ec4287357ad-data-volume\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.852939 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.852941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aea20af8-ca7e-4d2f-af08-6ec4287357ad-crio-socket\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.853160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.852967 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87sv\" (UniqueName: \"kubernetes.io/projected/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-api-access-x87sv\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.853160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.853000 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.853160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.853063 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aea20af8-ca7e-4d2f-af08-6ec4287357ad-crio-socket\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.853160 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.853080 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.853160 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:10.853156 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:10.853463 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:10.853261 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:10.853463 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:10.853287 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:11.353265318 +0000 UTC m=+267.010372796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : configmap "kube-rbac-proxy" not found Jul 08 19:46:10.853463 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.853304 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aea20af8-ca7e-4d2f-af08-6ec4287357ad-data-volume\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:10.853463 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:10.853313 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:11.353297853 +0000 UTC m=+267.010405312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:10.862438 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:10.862412 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87sv\" (UniqueName: \"kubernetes.io/projected/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-api-access-x87sv\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:11.357482 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:11.357437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:11.357665 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:11.357545 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:11.357665 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:11.357573 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:11.357665 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:11.357650 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:11.357760 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:11.357665 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:12.357641886 +0000 UTC m=+268.014749348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : configmap "kube-rbac-proxy" not found Jul 08 19:46:11.357760 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:11.357694 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:12.357684127 +0000 UTC m=+268.014791586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:12.364828 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:12.364787 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:12.365342 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:12.364865 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:12.365342 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:12.364931 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:12.365342 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:12.364986 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:12.365342 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:12.365026 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:14.365005728 +0000 UTC m=+270.022113188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : configmap "kube-rbac-proxy" not found Jul 08 19:46:12.365342 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:12.365046 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:14.365036629 +0000 UTC m=+270.022144089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:12.493802 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:12.493775 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6phpd_3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84/dns-node-resolver/0.log" Jul 08 19:46:13.694943 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:13.694911 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ppvcz_9a05fefd-7dcf-4fdc-833e-6ee404a4de0f/node-ca/0.log" Jul 08 19:46:14.293970 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:14.293940 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-24rzx_c24c75a7-a8f5-4534-85e6-8f9431128780/serve-healthcheck-canary/0.log" Jul 08 19:46:14.381611 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:14.381573 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:14.381793 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:14.381665 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:14.381793 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:14.381731 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:14.381793 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:14.381771 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:14.381908 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:14.381804 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:18.381787089 +0000 UTC m=+274.038894569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:14.381908 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:14.381822 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:18.381814709 +0000 UTC m=+274.038922167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : configmap "kube-rbac-proxy" not found Jul 08 19:46:18.416570 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:18.416533 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:18.416954 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:18.416613 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:18.416954 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:18.416653 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:46:18.416954 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:18.416707 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:46:18.416954 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:18.416720 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:26.416703968 +0000 UTC m=+282.073811427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : configmap "kube-rbac-proxy" not found Jul 08 19:46:18.416954 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:18.416749 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls podName:aea20af8-ca7e-4d2f-af08-6ec4287357ad nodeName:}" failed. No retries permitted until 2026-07-08 19:46:26.416738798 +0000 UTC m=+282.073846257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sgkx5" (UID: "aea20af8-ca7e-4d2f-af08-6ec4287357ad") : secret "insights-runtime-extractor-tls" not found Jul 08 19:46:26.483413 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.483365 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:26.483803 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.483463 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:26.483880 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.483860 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aea20af8-ca7e-4d2f-af08-6ec4287357ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:26.485630 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.485613 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aea20af8-ca7e-4d2f-af08-6ec4287357ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sgkx5\" (UID: \"aea20af8-ca7e-4d2f-af08-6ec4287357ad\") " pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:26.594953 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.594905 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sgkx5" Jul 08 19:46:26.713875 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.713844 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sgkx5"] Jul 08 19:46:26.717242 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:46:26.717214 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea20af8_ca7e_4d2f_af08_6ec4287357ad.slice/crio-bca2c32343fc5d303a342fdf826b474bc2992e2166a0ab2779f9ce3c411eb537 WatchSource:0}: Error finding container bca2c32343fc5d303a342fdf826b474bc2992e2166a0ab2779f9ce3c411eb537: Status 404 returned error can't find the container with id bca2c32343fc5d303a342fdf826b474bc2992e2166a0ab2779f9ce3c411eb537 Jul 08 19:46:26.768032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:26.767954 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sgkx5" event={"ID":"aea20af8-ca7e-4d2f-af08-6ec4287357ad","Type":"ContainerStarted","Data":"bca2c32343fc5d303a342fdf826b474bc2992e2166a0ab2779f9ce3c411eb537"} Jul 08 19:46:29.416465 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:29.416384 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" Jul 08 19:46:29.416465 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:29.416397 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" podUID="311603ab-67b8-4d51-92b9-2522a8e68cb3" Jul 08 19:46:29.416465 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:29.416399 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-r88l4" podUID="ec911e95-6f22-4710-9bf4-d3ebf667c684" Jul 08 19:46:29.778021 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:29.777939 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r88l4" Jul 08 19:46:29.778188 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:29.777939 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:46:29.778188 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:29.777939 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:46:30.782727 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:30.782645 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sgkx5" event={"ID":"aea20af8-ca7e-4d2f-af08-6ec4287357ad","Type":"ContainerStarted","Data":"3990c622ec30b228eee8ea27fcaf61563145c2a335d0677dc27aec0faceac08b"} Jul 08 19:46:31.788216 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:31.788160 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sgkx5" event={"ID":"aea20af8-ca7e-4d2f-af08-6ec4287357ad","Type":"ContainerStarted","Data":"f365d290c5ed98db74484e35c454cd273189325fad6a07d5aa3774c4db0e56e2"} Jul 08 19:46:33.241721 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.241679 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:46:33.242145 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.241747 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:46:33.242145 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.241788 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:46:33.244480 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.244453 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/311603ab-67b8-4d51-92b9-2522a8e68cb3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-bfbrm\" (UID: \"311603ab-67b8-4d51-92b9-2522a8e68cb3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:46:33.244674 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.244654 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec911e95-6f22-4710-9bf4-d3ebf667c684-metrics-tls\") pod \"dns-default-r88l4\" (UID: \"ec911e95-6f22-4710-9bf4-d3ebf667c684\") " pod="openshift-dns/dns-default-r88l4" Jul 08 19:46:33.244830 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.244799 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"image-registry-59586d4b44-92bwp\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:46:33.383022 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.382984 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w5s5j\"" Jul 08 19:46:33.384462 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.384434 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4fdwr\"" Jul 08 19:46:33.384590 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.384471 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z9rcf\"" Jul 08 19:46:33.390420 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.390383 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" Jul 08 19:46:33.390542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.390451 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:46:33.390542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.390393 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r88l4" Jul 08 19:46:33.596246 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.596163 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59586d4b44-92bwp"] Jul 08 19:46:33.599665 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:46:33.599636 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3e3326_af03_43a2_a7d0_b91d79924d6c.slice/crio-c3c755b30e44ad2e10dafb5ec65334a6024528aeb199473432d4ab5bf5cfc790 WatchSource:0}: Error finding container c3c755b30e44ad2e10dafb5ec65334a6024528aeb199473432d4ab5bf5cfc790: Status 404 returned error can't find the container with id c3c755b30e44ad2e10dafb5ec65334a6024528aeb199473432d4ab5bf5cfc790 Jul 08 19:46:33.609298 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.609235 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r88l4"] Jul 08 19:46:33.612477 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:46:33.612452 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec911e95_6f22_4710_9bf4_d3ebf667c684.slice/crio-ea5bb3e179da1cba487de6436c5946675ebf4b8a3bc0a1a325c27bccda428753 WatchSource:0}: Error finding container ea5bb3e179da1cba487de6436c5946675ebf4b8a3bc0a1a325c27bccda428753: Status 404 returned error can't find the container with id ea5bb3e179da1cba487de6436c5946675ebf4b8a3bc0a1a325c27bccda428753 Jul 08 19:46:33.621809 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.621770 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm"] Jul 08 19:46:33.630094 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:46:33.630070 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311603ab_67b8_4d51_92b9_2522a8e68cb3.slice/crio-29d745fa11dfe703404c21586065420e81fb07466d8b8bfd7bf71cdc97a5d066 WatchSource:0}: Error finding container 29d745fa11dfe703404c21586065420e81fb07466d8b8bfd7bf71cdc97a5d066: Status 404 returned error can't find the container with id 29d745fa11dfe703404c21586065420e81fb07466d8b8bfd7bf71cdc97a5d066 Jul 08 19:46:33.797479 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.797387 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" event={"ID":"fa3e3326-af03-43a2-a7d0-b91d79924d6c","Type":"ContainerStarted","Data":"90e60b4efa6742fac5a2ab54c0d01225940076820268fbb762a78d2debc36bd5"} Jul 08 19:46:33.797479 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.797441 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" event={"ID":"fa3e3326-af03-43a2-a7d0-b91d79924d6c","Type":"ContainerStarted","Data":"c3c755b30e44ad2e10dafb5ec65334a6024528aeb199473432d4ab5bf5cfc790"} Jul 08 19:46:33.797686 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.797508 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:46:33.799487 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.799453 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sgkx5" event={"ID":"aea20af8-ca7e-4d2f-af08-6ec4287357ad","Type":"ContainerStarted","Data":"189f6436e27b97db9005303b5097b4ed4d75fd33ff8e51341b4f0b795bb4eaac"} Jul 08 19:46:33.800634 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.800608 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" event={"ID":"311603ab-67b8-4d51-92b9-2522a8e68cb3","Type":"ContainerStarted","Data":"29d745fa11dfe703404c21586065420e81fb07466d8b8bfd7bf71cdc97a5d066"} Jul 08 19:46:33.801608 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.801591 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r88l4" event={"ID":"ec911e95-6f22-4710-9bf4-d3ebf667c684","Type":"ContainerStarted","Data":"ea5bb3e179da1cba487de6436c5946675ebf4b8a3bc0a1a325c27bccda428753"} Jul 08 19:46:33.820709 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.820660 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" podStartSLOduration=288.820649045 podStartE2EDuration="4m48.820649045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:33.820107591 +0000 UTC m=+289.477215075" watchObservedRunningTime="2026-07-08 19:46:33.820649045 +0000 UTC m=+289.477756525" Jul 08 19:46:33.841992 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:33.841947 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sgkx5" podStartSLOduration=17.17656582 podStartE2EDuration="23.841931321s" totalImagesPullingTime="6.665365501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:46:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:33.841329691 +0000 UTC m=+289.498437171" watchObservedRunningTime="2026-07-08 19:46:33.841931321 +0000 UTC m=+289.499038783" Jul 08 19:46:34.806434 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:34.806389 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" event={"ID":"311603ab-67b8-4d51-92b9-2522a8e68cb3","Type":"ContainerStarted","Data":"eaf80715880ce4185cec20d0dc91e798a3ae8b686b4e95e458641150a599ce8b"} Jul 08 19:46:34.821255 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:34.821182 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-bfbrm" podStartSLOduration=276.772817744 podStartE2EDuration="4m37.821165977s" totalImagesPullingTime="1.048348233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:34.820815213 +0000 UTC m=+290.477922694" watchObservedRunningTime="2026-07-08 19:46:34.821165977 +0000 UTC m=+290.478273459" Jul 08 19:46:35.811562 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.811528 2569 generic.go:356] "Generic (PLEG): container finished" podID="7e9004e7-9f85-408f-b1ca-f2deaa0f82f7" containerID="9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c" exitCode=1 Jul 08 19:46:35.812025 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.811597 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" event={"ID":"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7","Type":"ContainerDied","Data":"9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c"} Jul 08 19:46:35.812025 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.811650 2569 scope.go:117] "RemoveContainer" containerID="2d09c101d3b6528b0c723be2ca8c1f4441d75540990b57e92e1848d189e1db5d" Jul 08 19:46:35.812025 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.811968 2569 scope.go:117] "RemoveContainer" containerID="9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c" Jul 08 19:46:35.812180 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:35.812163 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-5d747fbc46-fs9vg_open-cluster-management-agent-addon(7e9004e7-9f85-408f-b1ca-f2deaa0f82f7)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" podUID="7e9004e7-9f85-408f-b1ca-f2deaa0f82f7" Jul 08 19:46:35.813505 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.813485 2569 generic.go:356] "Generic (PLEG): container finished" podID="4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd" containerID="dfcf4527528dbce1d7dba2054cc661a31bdc5d34da602c7d1f69369cf7d485dc" exitCode=255 Jul 08 19:46:35.813589 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.813555 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" event={"ID":"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd","Type":"ContainerDied","Data":"dfcf4527528dbce1d7dba2054cc661a31bdc5d34da602c7d1f69369cf7d485dc"} Jul 08 19:46:35.813883 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.813865 2569 scope.go:117] "RemoveContainer" containerID="dfcf4527528dbce1d7dba2054cc661a31bdc5d34da602c7d1f69369cf7d485dc" Jul 08 19:46:35.814020 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:35.814001 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-674b855b74-xtx4t_open-cluster-management-agent-addon(4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" podUID="4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd" Jul 08 19:46:35.815217 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.815185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r88l4" event={"ID":"ec911e95-6f22-4710-9bf4-d3ebf667c684","Type":"ContainerStarted","Data":"cd882761ba043654b31b1f7f49bc6e8eb4ffbbd0450ab0ba3edb3bdadb47200a"} Jul 08 19:46:35.815318 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.815238 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r88l4" event={"ID":"ec911e95-6f22-4710-9bf4-d3ebf667c684","Type":"ContainerStarted","Data":"2fb3d197871274295bc61644974c16178bf07c622b786422ba4fcefb16f7822d"} Jul 08 19:46:35.821699 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.821684 2569 scope.go:117] "RemoveContainer" containerID="f141fcddcd845e64d5024360a384936365c96f707ee335cb68bf1e38d3bb44e6" Jul 08 19:46:35.851472 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:35.851423 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r88l4" podStartSLOduration=250.893675386 podStartE2EDuration="4m12.851404091s" totalImagesPullingTime="1.957728705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:35.850730258 +0000 UTC m=+291.507837764" watchObservedRunningTime="2026-07-08 19:46:35.851404091 +0000 UTC m=+291.508511572" Jul 08 19:46:36.449740 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:36.449695 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:46:36.820057 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:36.820032 2569 scope.go:117] "RemoveContainer" containerID="9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c" Jul 08 19:46:36.820515 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:36.820225 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-5d747fbc46-fs9vg_open-cluster-management-agent-addon(7e9004e7-9f85-408f-b1ca-f2deaa0f82f7)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" podUID="7e9004e7-9f85-408f-b1ca-f2deaa0f82f7" Jul 08 19:46:36.821392 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:36.821375 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r88l4" Jul 08 19:46:39.387305 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:39.387248 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" Jul 08 19:46:39.387704 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:39.387618 2569 scope.go:117] "RemoveContainer" containerID="dfcf4527528dbce1d7dba2054cc661a31bdc5d34da602c7d1f69369cf7d485dc" Jul 08 19:46:39.387784 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:39.387763 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-674b855b74-xtx4t_open-cluster-management-agent-addon(4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" podUID="4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd" Jul 08 19:46:39.424308 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:39.424271 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:46:39.424644 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:39.424631 2569 scope.go:117] "RemoveContainer" containerID="9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c" Jul 08 19:46:39.424817 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:39.424796 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-5d747fbc46-fs9vg_open-cluster-management-agent-addon(7e9004e7-9f85-408f-b1ca-f2deaa0f82f7)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" podUID="7e9004e7-9f85-408f-b1ca-f2deaa0f82f7" Jul 08 19:46:44.737026 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:44.736997 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:46:45.852914 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:45.852874 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dvcrj" podUID="42b16c47-efdc-4d44-9501-99711b3e6487" Jul 08 19:46:46.830457 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:46.830429 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r88l4" Jul 08 19:46:50.853338 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:50.853307 2569 scope.go:117] "RemoveContainer" containerID="9bf4b9e07e0dcdfca6062b0332f3faf60e8f9ad716bbf94a4695dab2d1ce7d9c" Jul 08 19:46:51.873641 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:51.873605 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" event={"ID":"7e9004e7-9f85-408f-b1ca-f2deaa0f82f7","Type":"ContainerStarted","Data":"afe230546f251a3e54269db76d74f8a250ab2b58b7ab31c874d76ccf637fea3a"} Jul 08 19:46:51.874070 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:51.873884 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:46:51.875274 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:51.875252 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d747fbc46-fs9vg" Jul 08 19:46:53.395491 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:53.395455 2569 patch_prober.go:28] interesting pod/image-registry-59586d4b44-92bwp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 19:46:53.395865 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:53.395513 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:46:53.852458 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:53.852427 2569 scope.go:117] "RemoveContainer" containerID="dfcf4527528dbce1d7dba2054cc661a31bdc5d34da602c7d1f69369cf7d485dc" Jul 08 19:46:54.813009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:54.812981 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:46:54.890091 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:54.890054 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-674b855b74-xtx4t" event={"ID":"4b47cdc1-aa7f-48c9-a301-af2e9b34ebdd","Type":"ContainerStarted","Data":"63c95f6e6f164c0d6dc656d126433ab24c3d293e68217d18033583f62d4bfd23"} Jul 08 19:46:55.734234 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:55.734181 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59586d4b44-92bwp"] Jul 08 19:46:58.060630 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.060597 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jwrp8"] Jul 08 19:46:58.067648 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.067628 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.076170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076147 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:46:58.076312 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076178 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:46:58.076312 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076193 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:46:58.076312 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076178 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:46:58.076312 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076248 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:46:58.076312 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076178 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:46:58.076583 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.076568 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dzqw7\"" Jul 08 19:46:58.223743 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.223695 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-root\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.223743 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.223742 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8dbc954-b59e-4665-a01a-e99be7e1bae6-metrics-client-ca\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.223973 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.223819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-sys\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.223973 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.223893 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-wtmp\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.223973 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.223920 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-textfile\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.224172 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.223972 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-accelerators-collector-config\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.224172 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.224001 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-tls\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.224172 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.224019 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr29k\" (UniqueName: \"kubernetes.io/projected/e8dbc954-b59e-4665-a01a-e99be7e1bae6-kube-api-access-jr29k\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.224172 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.224040 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.324634 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-textfile\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.324634 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324600 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-accelerators-collector-config\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.324634 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324621 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-tls\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.324893 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr29k\" (UniqueName: \"kubernetes.io/projected/e8dbc954-b59e-4665-a01a-e99be7e1bae6-kube-api-access-jr29k\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.324893 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324766 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.324893 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324882 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-root\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325041 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:58.324897 2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 19:46:58.325041 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.324913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8dbc954-b59e-4665-a01a-e99be7e1bae6-metrics-client-ca\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325041 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:46:58.324979 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-tls podName:e8dbc954-b59e-4665-a01a-e99be7e1bae6 nodeName:}" failed. No retries permitted until 2026-07-08 19:46:58.824958308 +0000 UTC m=+314.482065769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-tls") pod "node-exporter-jwrp8" (UID: "e8dbc954-b59e-4665-a01a-e99be7e1bae6") : secret "node-exporter-tls" not found Jul 08 19:46:58.325041 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325012 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-textfile\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325041 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325024 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-sys\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325390 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325085 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-sys\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325390 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325086 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-wtmp\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325390 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325106 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-root\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325390 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325265 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-wtmp\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325390 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325307 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-accelerators-collector-config\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.325559 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.325419 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8dbc954-b59e-4665-a01a-e99be7e1bae6-metrics-client-ca\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.327117 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.327098 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.337103 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.337079 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr29k\" (UniqueName: \"kubernetes.io/projected/e8dbc954-b59e-4665-a01a-e99be7e1bae6-kube-api-access-jr29k\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.828813 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.828780 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-tls\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.831010 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.830977 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8dbc954-b59e-4665-a01a-e99be7e1bae6-node-exporter-tls\") pod \"node-exporter-jwrp8\" (UID: \"e8dbc954-b59e-4665-a01a-e99be7e1bae6\") " pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.976983 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.976929 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jwrp8" Jul 08 19:46:58.987531 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:46:58.987505 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dbc954_b59e_4665_a01a_e99be7e1bae6.slice/crio-00749789f9f6e0046d1abc059a7a764c48dfabf862a5a524dfb8d8b98383234a WatchSource:0}: Error finding container 00749789f9f6e0046d1abc059a7a764c48dfabf862a5a524dfb8d8b98383234a: Status 404 returned error can't find the container with id 00749789f9f6e0046d1abc059a7a764c48dfabf862a5a524dfb8d8b98383234a Jul 08 19:46:58.989512 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:58.989494 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:46:59.852425 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:59.852405 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:46:59.906245 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:59.906179 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jwrp8" event={"ID":"e8dbc954-b59e-4665-a01a-e99be7e1bae6","Type":"ContainerStarted","Data":"26db9021fc78ff4e648007c2f04b80fb42cb3ede55ce903d2f9c54831bdd39a8"} Jul 08 19:46:59.906362 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:46:59.906261 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jwrp8" event={"ID":"e8dbc954-b59e-4665-a01a-e99be7e1bae6","Type":"ContainerStarted","Data":"00749789f9f6e0046d1abc059a7a764c48dfabf862a5a524dfb8d8b98383234a"} Jul 08 19:47:00.910408 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:00.910374 2569 generic.go:356] "Generic (PLEG): container finished" podID="e8dbc954-b59e-4665-a01a-e99be7e1bae6" containerID="26db9021fc78ff4e648007c2f04b80fb42cb3ede55ce903d2f9c54831bdd39a8" exitCode=0 Jul 08 19:47:00.910408 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:00.910411 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jwrp8" event={"ID":"e8dbc954-b59e-4665-a01a-e99be7e1bae6","Type":"ContainerDied","Data":"26db9021fc78ff4e648007c2f04b80fb42cb3ede55ce903d2f9c54831bdd39a8"} Jul 08 19:47:01.915481 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:01.915448 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jwrp8" event={"ID":"e8dbc954-b59e-4665-a01a-e99be7e1bae6","Type":"ContainerStarted","Data":"55eaba7c2599cfc76a7a2fc18e912d7c9c0267376dd3e3aa7c21fcbe66197dff"} Jul 08 19:47:01.915876 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:01.915515 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jwrp8" event={"ID":"e8dbc954-b59e-4665-a01a-e99be7e1bae6","Type":"ContainerStarted","Data":"4c96cb50acf3a5c1fb52e34a50b45492c806a8f4a0cbcc12c5aa46f1de2ea25f"} Jul 08 19:47:01.937699 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:01.937649 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jwrp8" podStartSLOduration=3.030973494 podStartE2EDuration="3.93762961s" totalImagesPullingTime="840.967111ms" totalInitContainerRuntime="65.689005ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:47:01.936056999 +0000 UTC m=+317.593164480" watchObservedRunningTime="2026-07-08 19:47:01.93762961 +0000 UTC m=+317.594737089" Jul 08 19:47:19.395776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:19.395731 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" podUID="15f9d4fe-12ca-4421-a7ff-f6128ee3b099" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 19:47:20.754975 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.754930 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" containerName="registry" containerID="cri-o://90e60b4efa6742fac5a2ab54c0d01225940076820268fbb762a78d2debc36bd5" gracePeriod=30 Jul 08 19:47:20.978785 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.978750 2569 generic.go:356] "Generic (PLEG): container finished" podID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" containerID="90e60b4efa6742fac5a2ab54c0d01225940076820268fbb762a78d2debc36bd5" exitCode=0 Jul 08 19:47:20.978908 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.978826 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" event={"ID":"fa3e3326-af03-43a2-a7d0-b91d79924d6c","Type":"ContainerDied","Data":"90e60b4efa6742fac5a2ab54c0d01225940076820268fbb762a78d2debc36bd5"} Jul 08 19:47:20.978908 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.978892 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" event={"ID":"fa3e3326-af03-43a2-a7d0-b91d79924d6c","Type":"ContainerDied","Data":"c3c755b30e44ad2e10dafb5ec65334a6024528aeb199473432d4ab5bf5cfc790"} Jul 08 19:47:20.979018 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.978917 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3c755b30e44ad2e10dafb5ec65334a6024528aeb199473432d4ab5bf5cfc790" Jul 08 19:47:20.984879 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.984861 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:47:20.999985 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.999962 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htqbr\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-kube-api-access-htqbr\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000105 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:20.999996 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-installation-pull-secrets\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000105 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000035 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-trusted-ca\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000105 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000059 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-bound-sa-token\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000288 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000133 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-image-registry-private-configuration\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000288 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000181 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-certificates\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000288 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000222 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa3e3326-af03-43a2-a7d0-b91d79924d6c-ca-trust-extracted\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000288 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000257 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") pod \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\" (UID: \"fa3e3326-af03-43a2-a7d0-b91d79924d6c\") " Jul 08 19:47:21.000492 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000379 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:21.000552 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000531 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-trusted-ca\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.000653 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.000608 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:47:21.002753 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.002710 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:47:21.002753 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.002734 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:47:21.002915 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.002857 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:47:21.002974 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.002936 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:47:21.002974 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.002959 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-kube-api-access-htqbr" (OuterVolumeSpecName: "kube-api-access-htqbr") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "kube-api-access-htqbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:47:21.008909 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.008871 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa3e3326-af03-43a2-a7d0-b91d79924d6c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fa3e3326-af03-43a2-a7d0-b91d79924d6c" (UID: "fa3e3326-af03-43a2-a7d0-b91d79924d6c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:47:21.101293 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101255 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htqbr\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-kube-api-access-htqbr\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.101293 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101284 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-installation-pull-secrets\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.101293 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101294 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-bound-sa-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.101293 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101302 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa3e3326-af03-43a2-a7d0-b91d79924d6c-image-registry-private-configuration\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.101550 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101312 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-certificates\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.101550 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101321 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa3e3326-af03-43a2-a7d0-b91d79924d6c-ca-trust-extracted\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.101550 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.101330 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa3e3326-af03-43a2-a7d0-b91d79924d6c-registry-tls\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:47:21.982255 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:21.982225 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59586d4b44-92bwp" Jul 08 19:47:22.005561 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:22.005534 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59586d4b44-92bwp"] Jul 08 19:47:22.014514 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:22.014485 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59586d4b44-92bwp"] Jul 08 19:47:22.856334 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:22.856302 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" path="/var/lib/kubelet/pods/fa3e3326-af03-43a2-a7d0-b91d79924d6c/volumes" Jul 08 19:47:29.395619 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:29.395568 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" podUID="15f9d4fe-12ca-4421-a7ff-f6128ee3b099" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 19:47:39.395012 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:39.394926 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" podUID="15f9d4fe-12ca-4421-a7ff-f6128ee3b099" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 19:47:39.395387 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:39.395020 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" Jul 08 19:47:39.395525 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:39.395495 2569 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"46092b337ff86ea8ab580fc32026c46254c354736b31366d4a3fcddd7ee473ff"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 08 19:47:39.395561 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:39.395544 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" podUID="15f9d4fe-12ca-4421-a7ff-f6128ee3b099" containerName="service-proxy" containerID="cri-o://46092b337ff86ea8ab580fc32026c46254c354736b31366d4a3fcddd7ee473ff" gracePeriod=30 Jul 08 19:47:40.047031 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:40.046999 2569 generic.go:356] "Generic (PLEG): container finished" podID="15f9d4fe-12ca-4421-a7ff-f6128ee3b099" containerID="46092b337ff86ea8ab580fc32026c46254c354736b31366d4a3fcddd7ee473ff" exitCode=2 Jul 08 19:47:40.047195 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:40.047080 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" event={"ID":"15f9d4fe-12ca-4421-a7ff-f6128ee3b099","Type":"ContainerDied","Data":"46092b337ff86ea8ab580fc32026c46254c354736b31366d4a3fcddd7ee473ff"} Jul 08 19:47:40.047195 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:40.047129 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67fb889cd5-gtqgc" event={"ID":"15f9d4fe-12ca-4421-a7ff-f6128ee3b099","Type":"ContainerStarted","Data":"a79fa3280f1efe3e20d560d846b6dffe1e4a48d9ae9d661b6b1c6a0467d3eaae"} Jul 08 19:47:58.672535 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:58.672494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:47:58.674780 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:58.674760 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42b16c47-efdc-4d44-9501-99711b3e6487-metrics-certs\") pod \"network-metrics-daemon-dvcrj\" (UID: \"42b16c47-efdc-4d44-9501-99711b3e6487\") " pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:47:58.955930 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:58.955825 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dpn4j\"" Jul 08 19:47:58.963263 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:58.963243 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dvcrj" Jul 08 19:47:59.083000 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:59.082968 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dvcrj"] Jul 08 19:47:59.086305 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:47:59.086275 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b16c47_efdc_4d44_9501_99711b3e6487.slice/crio-68df4ccefc046776f84b9d3bba56b0737b8b42e425ef79115251e822ce9f3030 WatchSource:0}: Error finding container 68df4ccefc046776f84b9d3bba56b0737b8b42e425ef79115251e822ce9f3030: Status 404 returned error can't find the container with id 68df4ccefc046776f84b9d3bba56b0737b8b42e425ef79115251e822ce9f3030 Jul 08 19:47:59.111185 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:47:59.111157 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvcrj" event={"ID":"42b16c47-efdc-4d44-9501-99711b3e6487","Type":"ContainerStarted","Data":"68df4ccefc046776f84b9d3bba56b0737b8b42e425ef79115251e822ce9f3030"} Jul 08 19:48:01.120026 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:48:01.119992 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvcrj" event={"ID":"42b16c47-efdc-4d44-9501-99711b3e6487","Type":"ContainerStarted","Data":"446d995928498244925ef4539b071d586886003e70fd33336b7777d8349771ae"} Jul 08 19:48:01.120428 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:48:01.120045 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dvcrj" event={"ID":"42b16c47-efdc-4d44-9501-99711b3e6487","Type":"ContainerStarted","Data":"8538b521b4ebaed8da64cfff9902add5ad0b209ed6b3be65ec0f2c182f4bbfa4"} Jul 08 19:48:01.138100 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:48:01.138052 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dvcrj" podStartSLOduration=375.017386648 podStartE2EDuration="6m16.13803591s" totalImagesPullingTime="1.120649262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:48:01.136245593 +0000 UTC m=+376.793353079" watchObservedRunningTime="2026-07-08 19:48:01.13803591 +0000 UTC m=+376.795143388" Jul 08 19:49:50.402254 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.402195 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hl62k"] Jul 08 19:49:50.402660 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.402485 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" containerName="registry" Jul 08 19:49:50.402660 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.402497 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" containerName="registry" Jul 08 19:49:50.402660 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.402560 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa3e3326-af03-43a2-a7d0-b91d79924d6c" containerName="registry" Jul 08 19:49:50.405086 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.405067 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hl62k"] Jul 08 19:49:50.405158 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.405149 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.408087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.408065 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 19:49:50.408087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.408069 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-zcfkc\"" Jul 08 19:49:50.408302 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.408100 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 19:49:50.525764 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.525727 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnd4\" (UniqueName: \"kubernetes.io/projected/8d037454-b54e-4bd5-8836-662c9328da7b-kube-api-access-4hnd4\") pod \"cert-manager-cainjector-559cdd8c87-hl62k\" (UID: \"8d037454-b54e-4bd5-8836-662c9328da7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.525948 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.525770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d037454-b54e-4bd5-8836-662c9328da7b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hl62k\" (UID: \"8d037454-b54e-4bd5-8836-662c9328da7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.626157 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.626123 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnd4\" (UniqueName: \"kubernetes.io/projected/8d037454-b54e-4bd5-8836-662c9328da7b-kube-api-access-4hnd4\") pod \"cert-manager-cainjector-559cdd8c87-hl62k\" (UID: \"8d037454-b54e-4bd5-8836-662c9328da7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.626157 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.626163 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d037454-b54e-4bd5-8836-662c9328da7b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hl62k\" (UID: \"8d037454-b54e-4bd5-8836-662c9328da7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.639041 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.639007 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnd4\" (UniqueName: \"kubernetes.io/projected/8d037454-b54e-4bd5-8836-662c9328da7b-kube-api-access-4hnd4\") pod \"cert-manager-cainjector-559cdd8c87-hl62k\" (UID: \"8d037454-b54e-4bd5-8836-662c9328da7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.639231 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.639214 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d037454-b54e-4bd5-8836-662c9328da7b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hl62k\" (UID: \"8d037454-b54e-4bd5-8836-662c9328da7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.713127 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.713047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" Jul 08 19:49:50.846103 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:50.846071 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hl62k"] Jul 08 19:49:50.849016 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:49:50.848977 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d037454_b54e_4bd5_8836_662c9328da7b.slice/crio-fd54be6573647090b788db49dbc8a680ee59d6649e817f24de0619c53766f1c5 WatchSource:0}: Error finding container fd54be6573647090b788db49dbc8a680ee59d6649e817f24de0619c53766f1c5: Status 404 returned error can't find the container with id fd54be6573647090b788db49dbc8a680ee59d6649e817f24de0619c53766f1c5 Jul 08 19:49:51.490345 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:51.490307 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" event={"ID":"8d037454-b54e-4bd5-8836-662c9328da7b","Type":"ContainerStarted","Data":"fd54be6573647090b788db49dbc8a680ee59d6649e817f24de0619c53766f1c5"} Jul 08 19:49:54.507845 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:54.507807 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" event={"ID":"8d037454-b54e-4bd5-8836-662c9328da7b","Type":"ContainerStarted","Data":"0f79eb6f7a9a7d2cb913a04550958fb60d298195ee21673cd7d9eebe5affb469"} Jul 08 19:49:54.541039 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:49:54.540977 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hl62k" podStartSLOduration=1.351982406 podStartE2EDuration="4.540960224s" totalImagesPullingTime="3.188977818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:49:54.540561242 +0000 UTC m=+490.197668723" watchObservedRunningTime="2026-07-08 19:49:54.540960224 +0000 UTC m=+490.198067704" Jul 08 19:50:18.058924 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.058884 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj"] Jul 08 19:50:18.062173 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.062150 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.064901 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.064876 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vt2rp\"" Jul 08 19:50:18.065009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.064947 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:50:18.065009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.064998 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:50:18.065097 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.065037 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:50:18.065280 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.065266 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:50:18.075714 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.075692 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj"] Jul 08 19:50:18.221913 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.221876 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc425cb2-2580-49fa-a05a-f3f77586e1f1-tmp\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.222088 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.221922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc425cb2-2580-49fa-a05a-f3f77586e1f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.222088 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.221941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc425cb2-2580-49fa-a05a-f3f77586e1f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.222088 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.221960 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22fkd\" (UniqueName: \"kubernetes.io/projected/cc425cb2-2580-49fa-a05a-f3f77586e1f1-kube-api-access-22fkd\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.222088 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.222024 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc425cb2-2580-49fa-a05a-f3f77586e1f1-manifests\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.323030 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.322942 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc425cb2-2580-49fa-a05a-f3f77586e1f1-tmp\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.323030 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.322984 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc425cb2-2580-49fa-a05a-f3f77586e1f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.323030 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.323004 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc425cb2-2580-49fa-a05a-f3f77586e1f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.323345 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.323050 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22fkd\" (UniqueName: \"kubernetes.io/projected/cc425cb2-2580-49fa-a05a-f3f77586e1f1-kube-api-access-22fkd\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.323345 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.323087 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc425cb2-2580-49fa-a05a-f3f77586e1f1-manifests\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.323521 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.323487 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc425cb2-2580-49fa-a05a-f3f77586e1f1-manifests\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.325145 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.325115 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc425cb2-2580-49fa-a05a-f3f77586e1f1-tmp\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.325383 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.325365 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc425cb2-2580-49fa-a05a-f3f77586e1f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.325450 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.325397 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc425cb2-2580-49fa-a05a-f3f77586e1f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.337972 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.337947 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22fkd\" (UniqueName: \"kubernetes.io/projected/cc425cb2-2580-49fa-a05a-f3f77586e1f1-kube-api-access-22fkd\") pod \"opendatahub-operator-controller-manager-795d769c5f-4przj\" (UID: \"cc425cb2-2580-49fa-a05a-f3f77586e1f1\") " pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.372008 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.371977 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:18.505344 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.505317 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj"] Jul 08 19:50:18.507892 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:18.507855 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc425cb2_2580_49fa_a05a_f3f77586e1f1.slice/crio-19b718f7c297f163cfa7a5087f03726416abba5f5faf3f86d9be3f1a869a3c00 WatchSource:0}: Error finding container 19b718f7c297f163cfa7a5087f03726416abba5f5faf3f86d9be3f1a869a3c00: Status 404 returned error can't find the container with id 19b718f7c297f163cfa7a5087f03726416abba5f5faf3f86d9be3f1a869a3c00 Jul 08 19:50:18.589850 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:18.589762 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" event={"ID":"cc425cb2-2580-49fa-a05a-f3f77586e1f1","Type":"ContainerStarted","Data":"19b718f7c297f163cfa7a5087f03726416abba5f5faf3f86d9be3f1a869a3c00"} Jul 08 19:50:21.602554 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:21.602521 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" event={"ID":"cc425cb2-2580-49fa-a05a-f3f77586e1f1","Type":"ContainerStarted","Data":"d9a523c93ae57d56705feb65e0a93e20ad2ff8b212810f6e8752eb96d523ad66"} Jul 08 19:50:23.611801 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:23.611763 2569 generic.go:356] "Generic (PLEG): container finished" podID="cc425cb2-2580-49fa-a05a-f3f77586e1f1" containerID="d9a523c93ae57d56705feb65e0a93e20ad2ff8b212810f6e8752eb96d523ad66" exitCode=0 Jul 08 19:50:23.612196 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:23.611829 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" event={"ID":"cc425cb2-2580-49fa-a05a-f3f77586e1f1","Type":"ContainerDied","Data":"d9a523c93ae57d56705feb65e0a93e20ad2ff8b212810f6e8752eb96d523ad66"} Jul 08 19:50:24.617146 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:24.617106 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" event={"ID":"cc425cb2-2580-49fa-a05a-f3f77586e1f1","Type":"ContainerStarted","Data":"809e2f0f51dd17736e6c697308264e8b6e5667095019179957a86951c2b7528f"} Jul 08 19:50:24.617636 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:24.617386 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:24.642025 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:24.641967 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" podStartSLOduration=2.248831208 podStartE2EDuration="6.641953479s" totalImagesPullingTime="3.35287254s" totalInitContainerRuntime="1.040249731s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:24.640120865 +0000 UTC m=+520.297228344" watchObservedRunningTime="2026-07-08 19:50:24.641953479 +0000 UTC m=+520.299060960" Jul 08 19:50:35.623409 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:35.623381 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-795d769c5f-4przj" Jul 08 19:50:38.117171 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.117086 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp"] Jul 08 19:50:38.125109 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.125088 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.128252 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.128225 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:50:38.128361 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.128236 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:50:38.129714 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.129694 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 19:50:38.129819 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.129698 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 19:50:38.129819 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.129781 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dv96p\"" Jul 08 19:50:38.141611 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.141590 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp"] Jul 08 19:50:38.277429 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.277389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-tls-certs\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.277605 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.277449 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvg2z\" (UniqueName: \"kubernetes.io/projected/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-kube-api-access-nvg2z\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.277605 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.277470 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-tmp\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.282227 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.282189 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cp4q5"] Jul 08 19:50:38.285266 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.285251 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:38.290247 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.290225 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 19:50:38.290633 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.290618 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-67ptt\"" Jul 08 19:50:38.298394 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.298372 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cp4q5"] Jul 08 19:50:38.378467 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.378384 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvg2z\" (UniqueName: \"kubernetes.io/projected/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-kube-api-access-nvg2z\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.378467 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.378425 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-tmp\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.378665 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.378471 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-tls-certs\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.380828 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.380805 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-tmp\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.380966 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.380944 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-tls-certs\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.396983 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.396960 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvg2z\" (UniqueName: \"kubernetes.io/projected/6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f-kube-api-access-nvg2z\") pod \"kube-auth-proxy-779dfd48cc-ztcjp\" (UID: \"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f\") " pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.434323 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.434291 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" Jul 08 19:50:38.479622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.479589 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkg87\" (UniqueName: \"kubernetes.io/projected/6f1a4521-25c4-4bf9-b596-5ff89de23d05-kube-api-access-mkg87\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:38.479622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.479626 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1a4521-25c4-4bf9-b596-5ff89de23d05-cert\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:38.562085 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.562055 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp"] Jul 08 19:50:38.564376 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:38.564348 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6961e0d3_088a_4bc9_8ab7_c4e1f9f7c25f.slice/crio-bd93b95ed18ac5422afb29a0a0f5a005bcf68ddf7094d023d5715feafd81a3dd WatchSource:0}: Error finding container bd93b95ed18ac5422afb29a0a0f5a005bcf68ddf7094d023d5715feafd81a3dd: Status 404 returned error can't find the container with id bd93b95ed18ac5422afb29a0a0f5a005bcf68ddf7094d023d5715feafd81a3dd Jul 08 19:50:38.580719 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.580697 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkg87\" (UniqueName: \"kubernetes.io/projected/6f1a4521-25c4-4bf9-b596-5ff89de23d05-kube-api-access-mkg87\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:38.580837 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.580725 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1a4521-25c4-4bf9-b596-5ff89de23d05-cert\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:38.580837 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:50:38.580836 2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 19:50:38.580943 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:50:38.580881 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f1a4521-25c4-4bf9-b596-5ff89de23d05-cert podName:6f1a4521-25c4-4bf9-b596-5ff89de23d05 nodeName:}" failed. No retries permitted until 2026-07-08 19:50:39.080866604 +0000 UTC m=+534.737974062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6f1a4521-25c4-4bf9-b596-5ff89de23d05-cert") pod "odh-model-controller-7998c67dc4-cp4q5" (UID: "6f1a4521-25c4-4bf9-b596-5ff89de23d05") : secret "odh-model-controller-webhook-cert" not found Jul 08 19:50:38.590016 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.589996 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkg87\" (UniqueName: \"kubernetes.io/projected/6f1a4521-25c4-4bf9-b596-5ff89de23d05-kube-api-access-mkg87\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:38.666525 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:38.666441 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" event={"ID":"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f","Type":"ContainerStarted","Data":"bd93b95ed18ac5422afb29a0a0f5a005bcf68ddf7094d023d5715feafd81a3dd"} Jul 08 19:50:39.085217 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.085166 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1a4521-25c4-4bf9-b596-5ff89de23d05-cert\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:39.087534 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.087513 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1a4521-25c4-4bf9-b596-5ff89de23d05-cert\") pod \"odh-model-controller-7998c67dc4-cp4q5\" (UID: \"6f1a4521-25c4-4bf9-b596-5ff89de23d05\") " pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:39.195015 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.194972 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:39.339047 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.338981 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cp4q5"] Jul 08 19:50:39.342159 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:39.342122 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1a4521_25c4_4bf9_b596_5ff89de23d05.slice/crio-e205ad2bb71c37502f3dc8cfdb8ca8fcb992275773949fc156820c4b22dba88c WatchSource:0}: Error finding container e205ad2bb71c37502f3dc8cfdb8ca8fcb992275773949fc156820c4b22dba88c: Status 404 returned error can't find the container with id e205ad2bb71c37502f3dc8cfdb8ca8fcb992275773949fc156820c4b22dba88c Jul 08 19:50:39.477764 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.477731 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhngw"] Jul 08 19:50:39.480964 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.480948 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.485394 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.485372 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 19:50:39.485619 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.485604 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 19:50:39.489520 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.489503 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 19:50:39.492977 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.492961 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:50:39.493256 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.493241 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jtrmq\"" Jul 08 19:50:39.507287 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.507264 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhngw"] Jul 08 19:50:39.589555 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589468 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-token\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.589555 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589532 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.589785 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.589785 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589605 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.589785 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589699 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-cacerts\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.589785 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589748 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60474867-9960-48a7-a9e0-e46eb0dc446a-local-certs\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.589981 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.589801 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghlr\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-kube-api-access-9ghlr\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.672772 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.672733 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" event={"ID":"6f1a4521-25c4-4bf9-b596-5ff89de23d05","Type":"ContainerStarted","Data":"e205ad2bb71c37502f3dc8cfdb8ca8fcb992275773949fc156820c4b22dba88c"} Jul 08 19:50:39.690306 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.690306 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690309 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.690542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690342 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-cacerts\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.690542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690377 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60474867-9960-48a7-a9e0-e46eb0dc446a-local-certs\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.690542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690404 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghlr\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-kube-api-access-9ghlr\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.690542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690443 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-token\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.690542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.690492 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.691241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.691192 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.692880 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.692855 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-cacerts\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.692988 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.692902 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60474867-9960-48a7-a9e0-e46eb0dc446a-local-certs\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.692988 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.692961 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.692988 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.692975 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.698742 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.698715 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-token\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.698899 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.698880 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghlr\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-kube-api-access-9ghlr\") pod \"istiod-openshift-gateway-7574459656-bhngw\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.790788 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.790751 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:39.933947 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:39.933912 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhngw"] Jul 08 19:50:39.936087 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:39.936056 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60474867_9960_48a7_a9e0_e46eb0dc446a.slice/crio-c7dadca4ab80f8d81a1882ce827518e448aae4ca73c219835a92de1d7c27c6e1 WatchSource:0}: Error finding container c7dadca4ab80f8d81a1882ce827518e448aae4ca73c219835a92de1d7c27c6e1: Status 404 returned error can't find the container with id c7dadca4ab80f8d81a1882ce827518e448aae4ca73c219835a92de1d7c27c6e1 Jul 08 19:50:40.681859 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:40.681818 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" event={"ID":"60474867-9960-48a7-a9e0-e46eb0dc446a","Type":"ContainerStarted","Data":"c7dadca4ab80f8d81a1882ce827518e448aae4ca73c219835a92de1d7c27c6e1"} Jul 08 19:50:43.696066 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:43.696030 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" event={"ID":"6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f","Type":"ContainerStarted","Data":"5b601c7f5a91f5224065522728cdd43d07f7ef2aa8aa6c9227862ee788539ca7"} Jul 08 19:50:43.697624 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:43.697595 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" event={"ID":"6f1a4521-25c4-4bf9-b596-5ff89de23d05","Type":"ContainerStarted","Data":"44f78054e0e7bb8ec0a597a4cef8e826794a26868e0dfbaf5735d06e9701453c"} Jul 08 19:50:43.697788 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:43.697731 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:43.714861 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:43.714814 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-779dfd48cc-ztcjp" podStartSLOduration=1.371793099 podStartE2EDuration="5.714799828s" totalImagesPullingTime="4.343006729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:43.713305075 +0000 UTC m=+539.370412556" watchObservedRunningTime="2026-07-08 19:50:43.714799828 +0000 UTC m=+539.371907308" Jul 08 19:50:43.729995 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:43.729951 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" podStartSLOduration=2.11670222 podStartE2EDuration="5.729936829s" totalImagesPullingTime="3.613234609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:43.729050147 +0000 UTC m=+539.386157628" watchObservedRunningTime="2026-07-08 19:50:43.729936829 +0000 UTC m=+539.387044308" Jul 08 19:50:47.955836 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:47.955797 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:50:47.956131 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:47.955865 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:50:48.722463 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:48.722430 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" event={"ID":"60474867-9960-48a7-a9e0-e46eb0dc446a","Type":"ContainerStarted","Data":"a8d74ec8acc45451e95426f98aca8cfdd4a1f0f4ab663a71d775008b07f95f11"} Jul 08 19:50:48.722625 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:48.722490 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:48.757850 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:48.757800 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" podStartSLOduration=1.7402485269999999 podStartE2EDuration="9.75777957s" totalImagesPullingTime="8.017531043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:48.756039138 +0000 UTC m=+544.413146620" watchObservedRunningTime="2026-07-08 19:50:48.75777957 +0000 UTC m=+544.414887053" Jul 08 19:50:48.993554 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:48.993461 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s"] Jul 08 19:50:48.996373 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:48.996350 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.005320 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.005300 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8vmsj\"" Jul 08 19:50:49.025971 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.025948 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 19:50:49.027258 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.027234 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s"] Jul 08 19:50:49.050260 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.050227 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf"] Jul 08 19:50:49.053180 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.053156 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpbgk\" (UniqueName: \"kubernetes.io/projected/00e4bccf-cb88-4671-a47c-7c0a63c7e807-kube-api-access-fpbgk\") pod \"kserve-controller-manager-d6f4b65b8-j9m2s\" (UID: \"00e4bccf-cb88-4671-a47c-7c0a63c7e807\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.053279 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.053191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00e4bccf-cb88-4671-a47c-7c0a63c7e807-cert\") pod \"kserve-controller-manager-d6f4b65b8-j9m2s\" (UID: \"00e4bccf-cb88-4671-a47c-7c0a63c7e807\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.053439 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.053420 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:49.061218 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:50:49.058161 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"llmisvc-webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" type="*v1.Secret" Jul 08 19:50:49.061218 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:50:49.058446 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"llmisvc-controller-manager-dockercfg-p9hcv\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p9hcv\"" type="*v1.Secret" Jul 08 19:50:49.103656 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.103624 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf"] Jul 08 19:50:49.153675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.153639 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpbgk\" (UniqueName: \"kubernetes.io/projected/00e4bccf-cb88-4671-a47c-7c0a63c7e807-kube-api-access-fpbgk\") pod \"kserve-controller-manager-d6f4b65b8-j9m2s\" (UID: \"00e4bccf-cb88-4671-a47c-7c0a63c7e807\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.153675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.153680 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00e4bccf-cb88-4671-a47c-7c0a63c7e807-cert\") pod \"kserve-controller-manager-d6f4b65b8-j9m2s\" (UID: \"00e4bccf-cb88-4671-a47c-7c0a63c7e807\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.153915 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.153726 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4tcc\" (UniqueName: \"kubernetes.io/projected/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-kube-api-access-t4tcc\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:49.153915 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.153767 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:49.156100 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.156075 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00e4bccf-cb88-4671-a47c-7c0a63c7e807-cert\") pod \"kserve-controller-manager-d6f4b65b8-j9m2s\" (UID: \"00e4bccf-cb88-4671-a47c-7c0a63c7e807\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.176832 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.176802 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpbgk\" (UniqueName: \"kubernetes.io/projected/00e4bccf-cb88-4671-a47c-7c0a63c7e807-kube-api-access-fpbgk\") pod \"kserve-controller-manager-d6f4b65b8-j9m2s\" (UID: \"00e4bccf-cb88-4671-a47c-7c0a63c7e807\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.255009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.254922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:49.255009 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.255004 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4tcc\" (UniqueName: \"kubernetes.io/projected/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-kube-api-access-t4tcc\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:49.285868 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.285839 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4tcc\" (UniqueName: \"kubernetes.io/projected/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-kube-api-access-t4tcc\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:49.306566 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.306525 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:49.528800 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:49.528764 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e4bccf_cb88_4671_a47c_7c0a63c7e807.slice/crio-4f9d79b27f03f3424a8cd1ffbafe56e74394b4e88fe0525c1bb31fe2beb35ef2 WatchSource:0}: Error finding container 4f9d79b27f03f3424a8cd1ffbafe56e74394b4e88fe0525c1bb31fe2beb35ef2: Status 404 returned error can't find the container with id 4f9d79b27f03f3424a8cd1ffbafe56e74394b4e88fe0525c1bb31fe2beb35ef2 Jul 08 19:50:49.531271 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.531251 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s"] Jul 08 19:50:49.728090 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.728055 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" event={"ID":"00e4bccf-cb88-4671-a47c-7c0a63c7e807","Type":"ContainerStarted","Data":"4f9d79b27f03f3424a8cd1ffbafe56e74394b4e88fe0525c1bb31fe2beb35ef2"} Jul 08 19:50:49.729995 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.729967 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-bhngw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:50:49.730120 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:49.730025 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" podUID="60474867-9960-48a7-a9e0-e46eb0dc446a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:50:50.119014 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.118986 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p9hcv\"" Jul 08 19:50:50.255921 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:50:50.255884 2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: failed to sync secret cache: timed out waiting for the condition Jul 08 19:50:50.256088 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:50:50.255965 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-cert podName:ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4 nodeName:}" failed. No retries permitted until 2026-07-08 19:50:50.755948267 +0000 UTC m=+546.413055726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-cert") pod "llmisvc-controller-manager-777c6bd7d5-ggxkf" (UID: "ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4") : failed to sync secret cache: timed out waiting for the condition Jul 08 19:50:50.482428 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.482345 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt"] Jul 08 19:50:50.485729 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.485707 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.488884 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.488861 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rfl5h\"" Jul 08 19:50:50.508979 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.508951 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt"] Jul 08 19:50:50.513568 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.513545 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 19:50:50.563672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563640 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.563863 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563684 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26f3293a-8e17-4411-940d-48c3866be1fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.563863 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563716 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.563863 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563740 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26f3293a-8e17-4411-940d-48c3866be1fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.563863 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563774 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.564075 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26f3293a-8e17-4411-940d-48c3866be1fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.564075 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563912 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.564075 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.563948 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.564075 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.564033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24zh6\" (UniqueName: \"kubernetes.io/projected/26f3293a-8e17-4411-940d-48c3866be1fd-kube-api-access-24zh6\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665368 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24zh6\" (UniqueName: \"kubernetes.io/projected/26f3293a-8e17-4411-940d-48c3866be1fd-kube-api-access-24zh6\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665368 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665371 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665397 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26f3293a-8e17-4411-940d-48c3866be1fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665446 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26f3293a-8e17-4411-940d-48c3866be1fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665469 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665493 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26f3293a-8e17-4411-940d-48c3866be1fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665526 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665607 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665558 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665952 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665897 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.665952 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665936 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.666044 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.665974 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.666044 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.666008 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.666419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.666394 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26f3293a-8e17-4411-940d-48c3866be1fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.667809 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.667768 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26f3293a-8e17-4411-940d-48c3866be1fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.668024 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.668006 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26f3293a-8e17-4411-940d-48c3866be1fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.681449 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.681423 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26f3293a-8e17-4411-940d-48c3866be1fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.684944 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.684920 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24zh6\" (UniqueName: \"kubernetes.io/projected/26f3293a-8e17-4411-940d-48c3866be1fd-kube-api-access-24zh6\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678q97rt\" (UID: \"26f3293a-8e17-4411-940d-48c3866be1fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.766692 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.766600 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:50.769457 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.769428 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ggxkf\" (UID: \"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:50.798338 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.798304 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:50.865006 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.864769 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:50.953003 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:50.952968 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt"] Jul 08 19:50:50.956148 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:50.956108 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f3293a_8e17_4411_940d_48c3866be1fd.slice/crio-92db0b7798f6648f0bf8ea4263b5c8e43b1e2971df5ac929f8348d09b2b5cc2c WatchSource:0}: Error finding container 92db0b7798f6648f0bf8ea4263b5c8e43b1e2971df5ac929f8348d09b2b5cc2c: Status 404 returned error can't find the container with id 92db0b7798f6648f0bf8ea4263b5c8e43b1e2971df5ac929f8348d09b2b5cc2c Jul 08 19:50:51.040355 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:51.040274 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf"] Jul 08 19:50:51.046465 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:50:51.046436 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podad9fa3d9_3950_4e8d_8c3f_ca3e9f0671f4.slice/crio-9c92e6f9f2b67cbd7dceebd9494af66d738320735ba00ae650a70c4f6556a4d8 WatchSource:0}: Error finding container 9c92e6f9f2b67cbd7dceebd9494af66d738320735ba00ae650a70c4f6556a4d8: Status 404 returned error can't find the container with id 9c92e6f9f2b67cbd7dceebd9494af66d738320735ba00ae650a70c4f6556a4d8 Jul 08 19:50:51.738632 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:51.738594 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" event={"ID":"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4","Type":"ContainerStarted","Data":"9c92e6f9f2b67cbd7dceebd9494af66d738320735ba00ae650a70c4f6556a4d8"} Jul 08 19:50:51.740079 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:51.740053 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" event={"ID":"26f3293a-8e17-4411-940d-48c3866be1fd","Type":"ContainerStarted","Data":"92db0b7798f6648f0bf8ea4263b5c8e43b1e2971df5ac929f8348d09b2b5cc2c"} Jul 08 19:50:52.017695 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:52.017594 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhngw"] Jul 08 19:50:52.017858 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:52.017816 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" podUID="60474867-9960-48a7-a9e0-e46eb0dc446a" containerName="discovery" containerID="cri-o://a8d74ec8acc45451e95426f98aca8cfdd4a1f0f4ab663a71d775008b07f95f11" gracePeriod=30 Jul 08 19:50:52.018858 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:52.018834 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:52.748052 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:52.748016 2569 generic.go:356] "Generic (PLEG): container finished" podID="60474867-9960-48a7-a9e0-e46eb0dc446a" containerID="a8d74ec8acc45451e95426f98aca8cfdd4a1f0f4ab663a71d775008b07f95f11" exitCode=0 Jul 08 19:50:52.748519 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:52.748069 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" event={"ID":"60474867-9960-48a7-a9e0-e46eb0dc446a","Type":"ContainerDied","Data":"a8d74ec8acc45451e95426f98aca8cfdd4a1f0f4ab663a71d775008b07f95f11"} Jul 08 19:50:53.056787 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.056755 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:53.090462 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090433 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-token\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.090618 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090492 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-ca-configmap\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.090618 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090518 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-dns-cert\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.090732 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090636 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60474867-9960-48a7-a9e0-e46eb0dc446a-local-certs\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.090922 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090753 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ghlr\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-kube-api-access-9ghlr\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.090922 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090858 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:50:53.090922 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090913 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-kubeconfig\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.091098 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.090975 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-cacerts\") pod \"60474867-9960-48a7-a9e0-e46eb0dc446a\" (UID: \"60474867-9960-48a7-a9e0-e46eb0dc446a\") " Jul 08 19:50:53.091587 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.091352 2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-ca-configmap\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.093282 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.093235 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-kube-api-access-9ghlr" (OuterVolumeSpecName: "kube-api-access-9ghlr") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "kube-api-access-9ghlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:50:53.093642 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.093616 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-token" (OuterVolumeSpecName: "istio-token") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:50:53.094179 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.094152 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:50:53.094337 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.094175 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:50:53.094423 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.094362 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-cacerts" (OuterVolumeSpecName: "cacerts") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:50:53.094590 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.094545 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60474867-9960-48a7-a9e0-e46eb0dc446a-local-certs" (OuterVolumeSpecName: "local-certs") pod "60474867-9960-48a7-a9e0-e46eb0dc446a" (UID: "60474867-9960-48a7-a9e0-e46eb0dc446a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:50:53.192696 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.192665 2569 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-cacerts\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.192696 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.192694 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.192696 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.192706 2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-csr-dns-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.192906 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.192715 2569 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/60474867-9960-48a7-a9e0-e46eb0dc446a-local-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.192906 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.192724 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ghlr\" (UniqueName: \"kubernetes.io/projected/60474867-9960-48a7-a9e0-e46eb0dc446a-kube-api-access-9ghlr\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.192906 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.192733 2569 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60474867-9960-48a7-a9e0-e46eb0dc446a-istio-kubeconfig\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:50:53.754266 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.754236 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" Jul 08 19:50:53.754849 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.754261 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bhngw" event={"ID":"60474867-9960-48a7-a9e0-e46eb0dc446a","Type":"ContainerDied","Data":"c7dadca4ab80f8d81a1882ce827518e448aae4ca73c219835a92de1d7c27c6e1"} Jul 08 19:50:53.754849 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.754326 2569 scope.go:117] "RemoveContainer" containerID="a8d74ec8acc45451e95426f98aca8cfdd4a1f0f4ab663a71d775008b07f95f11" Jul 08 19:50:53.756881 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.756856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" event={"ID":"00e4bccf-cb88-4671-a47c-7c0a63c7e807","Type":"ContainerStarted","Data":"eecc69475824971a90392fb29ac7e4ff7ee6986625b9376a7741546fa7b2edcc"} Jul 08 19:50:53.757019 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.756997 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:50:53.789513 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.789457 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" podStartSLOduration=2.231323124 podStartE2EDuration="5.789435179s" totalImagesPullingTime="3.558112055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:53.788573978 +0000 UTC m=+549.445681459" watchObservedRunningTime="2026-07-08 19:50:53.789435179 +0000 UTC m=+549.446542661" Jul 08 19:50:53.818131 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.818098 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhngw"] Jul 08 19:50:53.847535 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:53.847505 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bhngw"] Jul 08 19:50:54.704321 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:54.704295 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cp4q5" Jul 08 19:50:54.859122 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:54.859087 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60474867-9960-48a7-a9e0-e46eb0dc446a" path="/var/lib/kubelet/pods/60474867-9960-48a7-a9e0-e46eb0dc446a/volumes" Jul 08 19:50:55.509491 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.509460 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:50:55.509611 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.509569 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:50:55.509674 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.509617 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:50:55.770679 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.770572 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" event={"ID":"ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4","Type":"ContainerStarted","Data":"5d4b5331c2e0c907bd2f639720a37233d7a0c16d97a3abebcc96f8cd3669b1c4"} Jul 08 19:50:55.770840 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.770686 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:50:55.772296 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.772275 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" event={"ID":"26f3293a-8e17-4411-940d-48c3866be1fd","Type":"ContainerStarted","Data":"4aaf4d657f5f96e4291066e0181788e6a39451e59c296686a45e0da0eb04afad"} Jul 08 19:50:55.798404 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.798375 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:55.799828 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.799741 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" podStartSLOduration=3.340583115 podStartE2EDuration="7.799725059s" totalImagesPullingTime="4.459141944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:55.798607877 +0000 UTC m=+551.455715370" watchObservedRunningTime="2026-07-08 19:50:55.799725059 +0000 UTC m=+551.456832538" Jul 08 19:50:55.831707 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:55.831654 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" podStartSLOduration=1.281606429 podStartE2EDuration="5.831634832s" totalImagesPullingTime="4.550028403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:50:55.831194669 +0000 UTC m=+551.488302148" watchObservedRunningTime="2026-07-08 19:50:55.831634832 +0000 UTC m=+551.488742315" Jul 08 19:50:56.803452 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:56.803421 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:57.780603 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:57.780563 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:50:57.781462 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:50:57.781444 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678q97rt" Jul 08 19:51:24.769252 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:24.769219 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j9m2s" Jul 08 19:51:26.779040 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:26.779009 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ggxkf" Jul 08 19:51:39.026021 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.025987 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hz4cr"] Jul 08 19:51:39.026478 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.026272 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60474867-9960-48a7-a9e0-e46eb0dc446a" containerName="discovery" Jul 08 19:51:39.026478 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.026282 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="60474867-9960-48a7-a9e0-e46eb0dc446a" containerName="discovery" Jul 08 19:51:39.026478 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.026334 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="60474867-9960-48a7-a9e0-e46eb0dc446a" containerName="discovery" Jul 08 19:51:39.038005 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.037971 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:39.040401 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.040372 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hz4cr"] Jul 08 19:51:39.041004 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.040978 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:51:39.041004 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.040989 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:51:39.041164 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.041070 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2cdxg\"" Jul 08 19:51:39.158087 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.158055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69656\" (UniqueName: \"kubernetes.io/projected/a91c3fb8-f8f4-4369-ad6b-179e4c024dde-kube-api-access-69656\") pod \"kuadrant-operator-catalog-hz4cr\" (UID: \"a91c3fb8-f8f4-4369-ad6b-179e4c024dde\") " pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:39.258771 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.258732 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69656\" (UniqueName: \"kubernetes.io/projected/a91c3fb8-f8f4-4369-ad6b-179e4c024dde-kube-api-access-69656\") pod \"kuadrant-operator-catalog-hz4cr\" (UID: \"a91c3fb8-f8f4-4369-ad6b-179e4c024dde\") " pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:39.266722 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.266684 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69656\" (UniqueName: \"kubernetes.io/projected/a91c3fb8-f8f4-4369-ad6b-179e4c024dde-kube-api-access-69656\") pod \"kuadrant-operator-catalog-hz4cr\" (UID: \"a91c3fb8-f8f4-4369-ad6b-179e4c024dde\") " pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:39.349388 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.349349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:39.396186 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.396153 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hz4cr"] Jul 08 19:51:39.477218 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.477180 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hz4cr"] Jul 08 19:51:39.481301 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:51:39.481263 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91c3fb8_f8f4_4369_ad6b_179e4c024dde.slice/crio-2b7f4a3c78d2404f5709a179d63458c54679dcbd7791a530a8c3e8a069f62a1a WatchSource:0}: Error finding container 2b7f4a3c78d2404f5709a179d63458c54679dcbd7791a530a8c3e8a069f62a1a: Status 404 returned error can't find the container with id 2b7f4a3c78d2404f5709a179d63458c54679dcbd7791a530a8c3e8a069f62a1a Jul 08 19:51:39.605949 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.605864 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xbf24"] Jul 08 19:51:39.609955 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.609937 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:39.617726 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.617703 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xbf24"] Jul 08 19:51:39.762402 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.762366 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s52xr\" (UniqueName: \"kubernetes.io/projected/7e372f94-8d03-4500-9eb3-24cc7df87f7c-kube-api-access-s52xr\") pod \"kuadrant-operator-catalog-xbf24\" (UID: \"7e372f94-8d03-4500-9eb3-24cc7df87f7c\") " pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:39.863702 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.863597 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s52xr\" (UniqueName: \"kubernetes.io/projected/7e372f94-8d03-4500-9eb3-24cc7df87f7c-kube-api-access-s52xr\") pod \"kuadrant-operator-catalog-xbf24\" (UID: \"7e372f94-8d03-4500-9eb3-24cc7df87f7c\") " pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:39.873663 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.873637 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s52xr\" (UniqueName: \"kubernetes.io/projected/7e372f94-8d03-4500-9eb3-24cc7df87f7c-kube-api-access-s52xr\") pod \"kuadrant-operator-catalog-xbf24\" (UID: \"7e372f94-8d03-4500-9eb3-24cc7df87f7c\") " pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:39.920264 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.920237 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:39.943113 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:39.943078 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" event={"ID":"a91c3fb8-f8f4-4369-ad6b-179e4c024dde","Type":"ContainerStarted","Data":"2b7f4a3c78d2404f5709a179d63458c54679dcbd7791a530a8c3e8a069f62a1a"} Jul 08 19:51:40.046035 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:40.045999 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xbf24"] Jul 08 19:51:40.048919 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:51:40.048892 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e372f94_8d03_4500_9eb3_24cc7df87f7c.slice/crio-4bb03fa0a9c5d0ed88524e8c8d628365108adb31fc61111b931d1aac97461128 WatchSource:0}: Error finding container 4bb03fa0a9c5d0ed88524e8c8d628365108adb31fc61111b931d1aac97461128: Status 404 returned error can't find the container with id 4bb03fa0a9c5d0ed88524e8c8d628365108adb31fc61111b931d1aac97461128 Jul 08 19:51:40.949029 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:40.948990 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" event={"ID":"7e372f94-8d03-4500-9eb3-24cc7df87f7c","Type":"ContainerStarted","Data":"4bb03fa0a9c5d0ed88524e8c8d628365108adb31fc61111b931d1aac97461128"} Jul 08 19:51:41.955602 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:41.955562 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" event={"ID":"a91c3fb8-f8f4-4369-ad6b-179e4c024dde","Type":"ContainerStarted","Data":"e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae"} Jul 08 19:51:41.956081 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:41.955650 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" podUID="a91c3fb8-f8f4-4369-ad6b-179e4c024dde" containerName="registry-server" containerID="cri-o://e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae" gracePeriod=2 Jul 08 19:51:41.957717 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:41.957353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" event={"ID":"7e372f94-8d03-4500-9eb3-24cc7df87f7c","Type":"ContainerStarted","Data":"a060a2154c123abf223a446fa565ba64fc8e628e47a764bb2fdbd516dc654d89"} Jul 08 19:51:41.971918 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:41.971870 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" podStartSLOduration=0.677163496 podStartE2EDuration="2.971853132s" totalImagesPullingTime="2.294689636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:51:41.970966081 +0000 UTC m=+597.628073565" watchObservedRunningTime="2026-07-08 19:51:41.971853132 +0000 UTC m=+597.628960650" Jul 08 19:51:41.987734 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:41.987686 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" podStartSLOduration=1.257349094 podStartE2EDuration="2.98767017s" totalImagesPullingTime="1.730321076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:51:41.985303998 +0000 UTC m=+597.642411477" watchObservedRunningTime="2026-07-08 19:51:41.98767017 +0000 UTC m=+597.644777649" Jul 08 19:51:42.190826 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.190801 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hz4cr_a91c3fb8-f8f4-4369-ad6b-179e4c024dde/registry-server/0.log" Jul 08 19:51:42.193090 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.193073 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:42.285668 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.285634 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69656\" (UniqueName: \"kubernetes.io/projected/a91c3fb8-f8f4-4369-ad6b-179e4c024dde-kube-api-access-69656\") pod \"a91c3fb8-f8f4-4369-ad6b-179e4c024dde\" (UID: \"a91c3fb8-f8f4-4369-ad6b-179e4c024dde\") " Jul 08 19:51:42.287859 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.287835 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91c3fb8-f8f4-4369-ad6b-179e4c024dde-kube-api-access-69656" (OuterVolumeSpecName: "kube-api-access-69656") pod "a91c3fb8-f8f4-4369-ad6b-179e4c024dde" (UID: "a91c3fb8-f8f4-4369-ad6b-179e4c024dde"). InnerVolumeSpecName "kube-api-access-69656". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:51:42.386525 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.386435 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69656\" (UniqueName: \"kubernetes.io/projected/a91c3fb8-f8f4-4369-ad6b-179e4c024dde-kube-api-access-69656\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:51:42.962254 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.962219 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hz4cr_a91c3fb8-f8f4-4369-ad6b-179e4c024dde/registry-server/0.log" Jul 08 19:51:42.962616 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.962261 2569 generic.go:356] "Generic (PLEG): container finished" podID="a91c3fb8-f8f4-4369-ad6b-179e4c024dde" containerID="e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae" exitCode=1 Jul 08 19:51:42.962616 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.962294 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" event={"ID":"a91c3fb8-f8f4-4369-ad6b-179e4c024dde","Type":"ContainerDied","Data":"e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae"} Jul 08 19:51:42.962616 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.962332 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" Jul 08 19:51:42.962616 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.962384 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hz4cr" event={"ID":"a91c3fb8-f8f4-4369-ad6b-179e4c024dde","Type":"ContainerDied","Data":"2b7f4a3c78d2404f5709a179d63458c54679dcbd7791a530a8c3e8a069f62a1a"} Jul 08 19:51:42.962616 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.962411 2569 scope.go:117] "RemoveContainer" containerID="e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae" Jul 08 19:51:42.970755 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.970739 2569 scope.go:117] "RemoveContainer" containerID="e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae" Jul 08 19:51:42.971049 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:51:42.971024 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae\": container with ID starting with e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae not found: ID does not exist" containerID="e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae" Jul 08 19:51:42.971118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.971056 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae"} err="failed to get container status \"e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae\": rpc error: code = NotFound desc = could not find container \"e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae\": container with ID starting with e7c2af2ab86191d72ab8f8ba34660e5ddd3f17993998fbdc6f5b3d381d570eae not found: ID does not exist" Jul 08 19:51:42.978733 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.978703 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hz4cr"] Jul 08 19:51:42.982364 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:42.982341 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hz4cr"] Jul 08 19:51:44.857403 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:44.857368 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91c3fb8-f8f4-4369-ad6b-179e4c024dde" path="/var/lib/kubelet/pods/a91c3fb8-f8f4-4369-ad6b-179e4c024dde/volumes" Jul 08 19:51:49.921384 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:49.921336 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:49.923745 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:49.921650 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:49.943815 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:49.943777 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:51:50.010825 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:51:50.010794 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xbf24" Jul 08 19:52:11.249081 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.248998 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g"] Jul 08 19:52:11.249560 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.249324 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91c3fb8-f8f4-4369-ad6b-179e4c024dde" containerName="registry-server" Jul 08 19:52:11.249560 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.249335 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91c3fb8-f8f4-4369-ad6b-179e4c024dde" containerName="registry-server" Jul 08 19:52:11.249560 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.249390 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91c3fb8-f8f4-4369-ad6b-179e4c024dde" containerName="registry-server" Jul 08 19:52:11.255530 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.255512 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:11.260576 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.260549 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xvt7x\"" Jul 08 19:52:11.268984 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.268959 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g"] Jul 08 19:52:11.308719 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.308679 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgcrw\" (UniqueName: \"kubernetes.io/projected/f93fd591-12bf-4bfd-9126-8c3cc5af6ff7-kube-api-access-lgcrw\") pod \"authorino-operator-75cc8d5cb7-lqj6g\" (UID: \"f93fd591-12bf-4bfd-9126-8c3cc5af6ff7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:11.409739 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.409696 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgcrw\" (UniqueName: \"kubernetes.io/projected/f93fd591-12bf-4bfd-9126-8c3cc5af6ff7-kube-api-access-lgcrw\") pod \"authorino-operator-75cc8d5cb7-lqj6g\" (UID: \"f93fd591-12bf-4bfd-9126-8c3cc5af6ff7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:11.421251 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.421220 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgcrw\" (UniqueName: \"kubernetes.io/projected/f93fd591-12bf-4bfd-9126-8c3cc5af6ff7-kube-api-access-lgcrw\") pod \"authorino-operator-75cc8d5cb7-lqj6g\" (UID: \"f93fd591-12bf-4bfd-9126-8c3cc5af6ff7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:11.565392 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.565362 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:11.714485 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.714394 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g"] Jul 08 19:52:11.717041 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:11.717008 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93fd591_12bf_4bfd_9126_8c3cc5af6ff7.slice/crio-ffc4dd71bca241d18755fcb0ce64d0ddfe31dfa197a23fce0a9eab0b2f747e70 WatchSource:0}: Error finding container ffc4dd71bca241d18755fcb0ce64d0ddfe31dfa197a23fce0a9eab0b2f747e70: Status 404 returned error can't find the container with id ffc4dd71bca241d18755fcb0ce64d0ddfe31dfa197a23fce0a9eab0b2f747e70 Jul 08 19:52:11.719585 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:11.719571 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:52:12.080785 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:12.080749 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" event={"ID":"f93fd591-12bf-4bfd-9126-8c3cc5af6ff7","Type":"ContainerStarted","Data":"ffc4dd71bca241d18755fcb0ce64d0ddfe31dfa197a23fce0a9eab0b2f747e70"} Jul 08 19:52:14.091083 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.091043 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" event={"ID":"f93fd591-12bf-4bfd-9126-8c3cc5af6ff7","Type":"ContainerStarted","Data":"6e316ba78968db16b5b36fe79b36b1f1b70306f1438fbc399d71a75c2f7fd545"} Jul 08 19:52:14.091516 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.091216 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:14.110340 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.110298 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" podStartSLOduration=0.8571105 podStartE2EDuration="3.110279849s" totalImagesPullingTime="2.253169349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:14.109311041 +0000 UTC m=+629.766418521" watchObservedRunningTime="2026-07-08 19:52:14.110279849 +0000 UTC m=+629.767387385" Jul 08 19:52:14.595100 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.595070 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9"] Jul 08 19:52:14.598195 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.598178 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:14.601967 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.601943 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 19:52:14.602330 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.602030 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cx2th\"" Jul 08 19:52:14.619882 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.619860 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9"] Jul 08 19:52:14.636455 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.636428 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcf9l\" (UniqueName: \"kubernetes.io/projected/8f489aa8-8c11-4ba8-8791-85d22b2965e8-kube-api-access-fcf9l\") pod \"dns-operator-controller-manager-864dd78c4c-6n4f9\" (UID: \"8f489aa8-8c11-4ba8-8791-85d22b2965e8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:14.737610 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.737577 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcf9l\" (UniqueName: \"kubernetes.io/projected/8f489aa8-8c11-4ba8-8791-85d22b2965e8-kube-api-access-fcf9l\") pod \"dns-operator-controller-manager-864dd78c4c-6n4f9\" (UID: \"8f489aa8-8c11-4ba8-8791-85d22b2965e8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:14.752587 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.752551 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcf9l\" (UniqueName: \"kubernetes.io/projected/8f489aa8-8c11-4ba8-8791-85d22b2965e8-kube-api-access-fcf9l\") pod \"dns-operator-controller-manager-864dd78c4c-6n4f9\" (UID: \"8f489aa8-8c11-4ba8-8791-85d22b2965e8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:14.908787 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:14.908701 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:15.046351 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:15.046325 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9"] Jul 08 19:52:15.048007 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:15.047981 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f489aa8_8c11_4ba8_8791_85d22b2965e8.slice/crio-e4da0e2be8f13066338ac079c9b6c5004d44c841d977735674515acbf099dca2 WatchSource:0}: Error finding container e4da0e2be8f13066338ac079c9b6c5004d44c841d977735674515acbf099dca2: Status 404 returned error can't find the container with id e4da0e2be8f13066338ac079c9b6c5004d44c841d977735674515acbf099dca2 Jul 08 19:52:15.096564 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:15.096524 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" event={"ID":"8f489aa8-8c11-4ba8-8791-85d22b2965e8","Type":"ContainerStarted","Data":"e4da0e2be8f13066338ac079c9b6c5004d44c841d977735674515acbf099dca2"} Jul 08 19:52:17.186632 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.186596 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7"] Jul 08 19:52:17.189978 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.189955 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:17.193622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.193604 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hrzqw\"" Jul 08 19:52:17.199395 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.199370 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7"] Jul 08 19:52:17.259274 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.259243 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9q26\" (UniqueName: \"kubernetes.io/projected/9b26c7d5-e069-4f89-b3d5-47a544f64b65-kube-api-access-p9q26\") pod \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" (UID: \"9b26c7d5-e069-4f89-b3d5-47a544f64b65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:17.359983 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.359947 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9q26\" (UniqueName: \"kubernetes.io/projected/9b26c7d5-e069-4f89-b3d5-47a544f64b65-kube-api-access-p9q26\") pod \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" (UID: \"9b26c7d5-e069-4f89-b3d5-47a544f64b65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:17.371344 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.371314 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9q26\" (UniqueName: \"kubernetes.io/projected/9b26c7d5-e069-4f89-b3d5-47a544f64b65-kube-api-access-p9q26\") pod \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" (UID: \"9b26c7d5-e069-4f89-b3d5-47a544f64b65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:17.501091 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.501003 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:17.832525 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:17.832354 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7"] Jul 08 19:52:17.835748 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:17.835715 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b26c7d5_e069_4f89_b3d5_47a544f64b65.slice/crio-100138d5a1d361812da08643f0b2d9a13732a8f7624c51a2a6e26e4215edde87 WatchSource:0}: Error finding container 100138d5a1d361812da08643f0b2d9a13732a8f7624c51a2a6e26e4215edde87: Status 404 returned error can't find the container with id 100138d5a1d361812da08643f0b2d9a13732a8f7624c51a2a6e26e4215edde87 Jul 08 19:52:18.110636 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:18.110602 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" event={"ID":"8f489aa8-8c11-4ba8-8791-85d22b2965e8","Type":"ContainerStarted","Data":"0292dbda8e1dbfbef2c5b583d489839f8473b928d7ee03d2bb54b564653c2dc5"} Jul 08 19:52:18.110846 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:18.110687 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:18.111880 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:18.111856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" event={"ID":"9b26c7d5-e069-4f89-b3d5-47a544f64b65","Type":"ContainerStarted","Data":"100138d5a1d361812da08643f0b2d9a13732a8f7624c51a2a6e26e4215edde87"} Jul 08 19:52:18.159187 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:18.159130 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" podStartSLOduration=1.468168891 podStartE2EDuration="4.159111276s" totalImagesPullingTime="2.690942385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:18.157621059 +0000 UTC m=+633.814728537" watchObservedRunningTime="2026-07-08 19:52:18.159111276 +0000 UTC m=+633.816218755" Jul 08 19:52:20.124112 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:20.124020 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" event={"ID":"9b26c7d5-e069-4f89-b3d5-47a544f64b65","Type":"ContainerStarted","Data":"efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41"} Jul 08 19:52:20.124590 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:20.124174 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:20.171975 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:20.171924 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" podStartSLOduration=1.162081285 podStartE2EDuration="3.171905237s" totalImagesPullingTime="2.009823952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:20.171087591 +0000 UTC m=+635.828195070" watchObservedRunningTime="2026-07-08 19:52:20.171905237 +0000 UTC m=+635.829012717" Jul 08 19:52:21.454467 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.454432 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd"] Jul 08 19:52:21.460027 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.459109 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.462786 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.462761 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gxwbh\"" Jul 08 19:52:21.470534 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.470509 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd"] Jul 08 19:52:21.494438 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.494406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fef02cb-1f5a-4cc1-9174-895400eb4e21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.494587 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.494446 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtr9q\" (UniqueName: \"kubernetes.io/projected/5fef02cb-1f5a-4cc1-9174-895400eb4e21-kube-api-access-xtr9q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.595749 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.595712 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fef02cb-1f5a-4cc1-9174-895400eb4e21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.595902 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.595757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtr9q\" (UniqueName: \"kubernetes.io/projected/5fef02cb-1f5a-4cc1-9174-895400eb4e21-kube-api-access-xtr9q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.596146 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.596123 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fef02cb-1f5a-4cc1-9174-895400eb4e21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.605219 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.605174 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtr9q\" (UniqueName: \"kubernetes.io/projected/5fef02cb-1f5a-4cc1-9174-895400eb4e21-kube-api-access-xtr9q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.769784 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.769693 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:21.901162 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:21.901130 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd"] Jul 08 19:52:21.904297 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:21.904268 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fef02cb_1f5a_4cc1_9174_895400eb4e21.slice/crio-203c557d01974d249c680aa95cf54ff94d1345882823019873a9970b24201d31 WatchSource:0}: Error finding container 203c557d01974d249c680aa95cf54ff94d1345882823019873a9970b24201d31: Status 404 returned error can't find the container with id 203c557d01974d249c680aa95cf54ff94d1345882823019873a9970b24201d31 Jul 08 19:52:22.133267 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:22.133233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" event={"ID":"5fef02cb-1f5a-4cc1-9174-895400eb4e21","Type":"ContainerStarted","Data":"203c557d01974d249c680aa95cf54ff94d1345882823019873a9970b24201d31"} Jul 08 19:52:25.098818 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:25.098782 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lqj6g" Jul 08 19:52:28.162578 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:28.162498 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" event={"ID":"5fef02cb-1f5a-4cc1-9174-895400eb4e21","Type":"ContainerStarted","Data":"c24e377fe0b27dea67c040ff8af94698d58cabb9e6e8da6c37462493db78662a"} Jul 08 19:52:28.162942 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:28.162643 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:28.190429 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:28.190370 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" podStartSLOduration=1.215181794 podStartE2EDuration="7.190352374s" totalImagesPullingTime="5.97517058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:28.188456035 +0000 UTC m=+643.845563520" watchObservedRunningTime="2026-07-08 19:52:28.190352374 +0000 UTC m=+643.847459853" Jul 08 19:52:29.119250 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:29.119220 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6n4f9" Jul 08 19:52:30.643295 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.643259 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6"] Jul 08 19:52:30.646830 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.646807 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.649896 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.649875 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 19:52:30.650033 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.649992 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xl2pr\"" Jul 08 19:52:30.650112 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.649997 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 19:52:30.656421 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.656399 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6"] Jul 08 19:52:30.770428 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.770391 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.770428 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.770428 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnnxx\" (UniqueName: \"kubernetes.io/projected/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-kube-api-access-gnnxx\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.770655 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.770452 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.871320 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.871284 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.871320 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.871322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnnxx\" (UniqueName: \"kubernetes.io/projected/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-kube-api-access-gnnxx\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.871542 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.871348 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.872005 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.871984 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.873757 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.873738 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.880219 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.880183 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnnxx\" (UniqueName: \"kubernetes.io/projected/e9a276d2-44fc-49e5-8ce4-ceda0a284f66-kube-api-access-gnnxx\") pod \"kuadrant-console-plugin-66b5774985-5v2h6\" (UID: \"e9a276d2-44fc-49e5-8ce4-ceda0a284f66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:30.956184 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:30.956095 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" Jul 08 19:52:31.087473 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:31.087443 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6"] Jul 08 19:52:31.089677 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:31.089653 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9a276d2_44fc_49e5_8ce4_ceda0a284f66.slice/crio-201f4af8c9c33f314060e8b80ad36328bd31344613ecd9b7fd8e099d19a38eed WatchSource:0}: Error finding container 201f4af8c9c33f314060e8b80ad36328bd31344613ecd9b7fd8e099d19a38eed: Status 404 returned error can't find the container with id 201f4af8c9c33f314060e8b80ad36328bd31344613ecd9b7fd8e099d19a38eed Jul 08 19:52:31.129780 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:31.129754 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:31.177075 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:31.177038 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" event={"ID":"e9a276d2-44fc-49e5-8ce4-ceda0a284f66","Type":"ContainerStarted","Data":"201f4af8c9c33f314060e8b80ad36328bd31344613ecd9b7fd8e099d19a38eed"} Jul 08 19:52:39.169104 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:39.169070 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.079017 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.078978 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd"] Jul 08 19:52:41.079453 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.079164 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" containerName="manager" containerID="cri-o://c24e377fe0b27dea67c040ff8af94698d58cabb9e6e8da6c37462493db78662a" gracePeriod=2 Jul 08 19:52:41.092787 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.092724 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd"] Jul 08 19:52:41.107762 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.107722 2569 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fef02cb-1f5a-4cc1-9174-895400eb4e21\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-d45jd\": pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.114675 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.114648 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st"] Jul 08 19:52:41.115012 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.114995 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" containerName="manager" Jul 08 19:52:41.115078 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.115015 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" containerName="manager" Jul 08 19:52:41.115133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.115119 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" containerName="manager" Jul 08 19:52:41.118003 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.117984 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7"] Jul 08 19:52:41.118114 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.118096 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.121277 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.118603 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" containerName="manager" containerID="cri-o://efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41" gracePeriod=2 Jul 08 19:52:41.126375 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.126340 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7"] Jul 08 19:52:41.126685 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.126659 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.128752 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.128715 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.26:8081/readyz\": dial tcp 10.132.0.26:8081: connect: connection refused" Jul 08 19:52:41.141118 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.141091 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st"] Jul 08 19:52:41.150126 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.150091 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt"] Jul 08 19:52:41.150405 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.150392 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" containerName="manager" Jul 08 19:52:41.150461 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.150406 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" containerName="manager" Jul 08 19:52:41.150500 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.150469 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" containerName="manager" Jul 08 19:52:41.153507 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.153478 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:41.166989 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.166955 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:41.169395 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.169372 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.185666 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.185597 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt"] Jul 08 19:52:41.200021 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.199890 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:41.202458 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.202414 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.228063 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.228036 2569 generic.go:356] "Generic (PLEG): container finished" podID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" containerID="c24e377fe0b27dea67c040ff8af94698d58cabb9e6e8da6c37462493db78662a" exitCode=0 Jul 08 19:52:41.262570 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.262529 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps9fc\" (UniqueName: \"kubernetes.io/projected/90f37c07-46cd-4185-b4e7-46a1491826e7-kube-api-access-ps9fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fz6st\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.262747 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.262573 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90f37c07-46cd-4185-b4e7-46a1491826e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fz6st\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.262747 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.262649 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbdr2\" (UniqueName: \"kubernetes.io/projected/a6a4ddd4-aa9b-4fce-900d-2a60f0045be5-kube-api-access-rbdr2\") pod \"limitador-operator-controller-manager-859d7b78bd-cjxlt\" (UID: \"a6a4ddd4-aa9b-4fce-900d-2a60f0045be5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:41.363532 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.363452 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90f37c07-46cd-4185-b4e7-46a1491826e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fz6st\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.363532 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.363504 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbdr2\" (UniqueName: \"kubernetes.io/projected/a6a4ddd4-aa9b-4fce-900d-2a60f0045be5-kube-api-access-rbdr2\") pod \"limitador-operator-controller-manager-859d7b78bd-cjxlt\" (UID: \"a6a4ddd4-aa9b-4fce-900d-2a60f0045be5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:41.363767 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.363564 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps9fc\" (UniqueName: \"kubernetes.io/projected/90f37c07-46cd-4185-b4e7-46a1491826e7-kube-api-access-ps9fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fz6st\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.363864 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.363841 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90f37c07-46cd-4185-b4e7-46a1491826e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fz6st\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.375969 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.375945 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps9fc\" (UniqueName: \"kubernetes.io/projected/90f37c07-46cd-4185-b4e7-46a1491826e7-kube-api-access-ps9fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fz6st\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.392233 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.392193 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:41.394708 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.394691 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.397144 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.397114 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:41.404998 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.404973 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.407912 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.407891 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:41.421938 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:41.421910 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:41.423093 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.423076 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbdr2\" (UniqueName: \"kubernetes.io/projected/a6a4ddd4-aa9b-4fce-900d-2a60f0045be5-kube-api-access-rbdr2\") pod \"limitador-operator-controller-manager-859d7b78bd-cjxlt\" (UID: \"a6a4ddd4-aa9b-4fce-900d-2a60f0045be5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:41.464029 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.463998 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtr9q\" (UniqueName: \"kubernetes.io/projected/5fef02cb-1f5a-4cc1-9174-895400eb4e21-kube-api-access-xtr9q\") pod \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " Jul 08 19:52:41.464167 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.464091 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fef02cb-1f5a-4cc1-9174-895400eb4e21-extensions-socket-volume\") pod \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\" (UID: \"5fef02cb-1f5a-4cc1-9174-895400eb4e21\") " Jul 08 19:52:41.464167 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.464127 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9q26\" (UniqueName: \"kubernetes.io/projected/9b26c7d5-e069-4f89-b3d5-47a544f64b65-kube-api-access-p9q26\") pod \"9b26c7d5-e069-4f89-b3d5-47a544f64b65\" (UID: \"9b26c7d5-e069-4f89-b3d5-47a544f64b65\") " Jul 08 19:52:41.464663 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.464622 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fef02cb-1f5a-4cc1-9174-895400eb4e21-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5fef02cb-1f5a-4cc1-9174-895400eb4e21" (UID: "5fef02cb-1f5a-4cc1-9174-895400eb4e21"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:41.466153 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.466131 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fef02cb-1f5a-4cc1-9174-895400eb4e21-kube-api-access-xtr9q" (OuterVolumeSpecName: "kube-api-access-xtr9q") pod "5fef02cb-1f5a-4cc1-9174-895400eb4e21" (UID: "5fef02cb-1f5a-4cc1-9174-895400eb4e21"). InnerVolumeSpecName "kube-api-access-xtr9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:52:41.466259 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.466161 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b26c7d5-e069-4f89-b3d5-47a544f64b65-kube-api-access-p9q26" (OuterVolumeSpecName: "kube-api-access-p9q26") pod "9b26c7d5-e069-4f89-b3d5-47a544f64b65" (UID: "9b26c7d5-e069-4f89-b3d5-47a544f64b65"). InnerVolumeSpecName "kube-api-access-p9q26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:52:41.507380 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.507332 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:41.515470 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.515444 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:41.565792 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.565419 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fef02cb-1f5a-4cc1-9174-895400eb4e21-extensions-socket-volume\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:52:41.565792 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.565456 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9q26\" (UniqueName: \"kubernetes.io/projected/9b26c7d5-e069-4f89-b3d5-47a544f64b65-kube-api-access-p9q26\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:52:41.565792 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.565475 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtr9q\" (UniqueName: \"kubernetes.io/projected/5fef02cb-1f5a-4cc1-9174-895400eb4e21-kube-api-access-xtr9q\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:52:41.676535 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.676455 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st"] Jul 08 19:52:41.679367 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:41.679334 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f37c07_46cd_4185_b4e7_46a1491826e7.slice/crio-30b746868e3193697afe0da547e16cfa8448c51f225951f85f534983659fc946 WatchSource:0}: Error finding container 30b746868e3193697afe0da547e16cfa8448c51f225951f85f534983659fc946: Status 404 returned error can't find the container with id 30b746868e3193697afe0da547e16cfa8448c51f225951f85f534983659fc946 Jul 08 19:52:41.695010 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:41.694989 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt"] Jul 08 19:52:41.697355 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:41.697328 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a4ddd4_aa9b_4fce_900d_2a60f0045be5.slice/crio-eb00a160a27e5bc6a15dd7fd7cd5db40e67a1e005f6b8eccfd57bd323ed97d5e WatchSource:0}: Error finding container eb00a160a27e5bc6a15dd7fd7cd5db40e67a1e005f6b8eccfd57bd323ed97d5e: Status 404 returned error can't find the container with id eb00a160a27e5bc6a15dd7fd7cd5db40e67a1e005f6b8eccfd57bd323ed97d5e Jul 08 19:52:42.235374 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.235333 2569 generic.go:356] "Generic (PLEG): container finished" podID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" containerID="efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41" exitCode=0 Jul 08 19:52:42.235901 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.235470 2569 scope.go:117] "RemoveContainer" containerID="efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41" Jul 08 19:52:42.235901 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.235619 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:42.238085 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.238036 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:42.238917 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:42.238886 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:42.240368 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.240337 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" event={"ID":"a6a4ddd4-aa9b-4fce-900d-2a60f0045be5","Type":"ContainerStarted","Data":"b8e1cee5a833c0380a8e85399e2f5eccdeafc5bdd624a6a2073d8189fc512ab8"} Jul 08 19:52:42.240476 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.240395 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" event={"ID":"a6a4ddd4-aa9b-4fce-900d-2a60f0045be5","Type":"ContainerStarted","Data":"eb00a160a27e5bc6a15dd7fd7cd5db40e67a1e005f6b8eccfd57bd323ed97d5e"} Jul 08 19:52:42.240825 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.240775 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:42.241619 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:42.241477 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:42.242848 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.242773 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" event={"ID":"90f37c07-46cd-4185-b4e7-46a1491826e7","Type":"ContainerStarted","Data":"1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc"} Jul 08 19:52:42.242848 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.242818 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" event={"ID":"90f37c07-46cd-4185-b4e7-46a1491826e7","Type":"ContainerStarted","Data":"30b746868e3193697afe0da547e16cfa8448c51f225951f85f534983659fc946"} Jul 08 19:52:42.243079 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.242959 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:42.244773 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:42.244744 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:42.251458 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:42.251434 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:42.277329 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.277278 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" podStartSLOduration=1.277260169 podStartE2EDuration="1.277260169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:42.276833442 +0000 UTC m=+657.933940922" watchObservedRunningTime="2026-07-08 19:52:42.277260169 +0000 UTC m=+657.934367649" Jul 08 19:52:42.318567 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.318319 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" podStartSLOduration=1.318295119 podStartE2EDuration="1.318295119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:42.318076424 +0000 UTC m=+657.975183905" watchObservedRunningTime="2026-07-08 19:52:42.318295119 +0000 UTC m=+657.975402599" Jul 08 19:52:42.322823 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:42.322785 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gp2p7\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gp2p7" Jul 08 19:52:42.325852 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:42.325815 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d45jd\" is forbidden: User \"system:node:ip-10-0-133-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-133.ec2.internal' and this object" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d45jd" Jul 08 19:52:42.859705 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.859672 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fef02cb-1f5a-4cc1-9174-895400eb4e21" path="/var/lib/kubelet/pods/5fef02cb-1f5a-4cc1-9174-895400eb4e21/volumes" Jul 08 19:52:42.860059 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:42.860042 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b26c7d5-e069-4f89-b3d5-47a544f64b65" path="/var/lib/kubelet/pods/9b26c7d5-e069-4f89-b3d5-47a544f64b65/volumes" Jul 08 19:52:44.767261 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:44.767231 2569 scope.go:117] "RemoveContainer" containerID="90e60b4efa6742fac5a2ab54c0d01225940076820268fbb762a78d2debc36bd5" Jul 08 19:52:53.250138 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:53.250106 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjxlt" Jul 08 19:52:53.250597 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:53.250190 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:56.168916 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:56.168632 2569 scope.go:117] "RemoveContainer" containerID="efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41" Jul 08 19:52:56.169253 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:56.168999 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41\": container with ID starting with efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41 not found: ID does not exist" containerID="efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41" Jul 08 19:52:56.169253 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:56.169032 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41"} err="failed to get container status \"efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41\": rpc error: code = NotFound desc = could not find container \"efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41\": container with ID starting with efa5d8199ae1cf6d0db0366fe89a41b34821f4dbe39403eb3d4e7ae3f86c7a41 not found: ID does not exist" Jul 08 19:52:56.169253 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:56.169054 2569 scope.go:117] "RemoveContainer" containerID="c24e377fe0b27dea67c040ff8af94698d58cabb9e6e8da6c37462493db78662a" Jul 08 19:52:56.318389 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:56.318362 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" event={"ID":"e9a276d2-44fc-49e5-8ce4-ceda0a284f66","Type":"ContainerStarted","Data":"538a362e84ab9c6cb07b701015de1fb61f7ba3267b9c32cab0be5c42c164200f"} Jul 08 19:52:56.336511 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:56.336467 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5v2h6" podStartSLOduration=1.177252309 podStartE2EDuration="26.336449952s" totalImagesPullingTime="25.159197643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:56.334864273 +0000 UTC m=+671.991971752" watchObservedRunningTime="2026-07-08 19:52:56.336449952 +0000 UTC m=+671.993557431" Jul 08 19:52:57.569830 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.569797 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st"] Jul 08 19:52:57.570267 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.570000 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" podUID="90f37c07-46cd-4185-b4e7-46a1491826e7" containerName="manager" containerID="cri-o://1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc" gracePeriod=10 Jul 08 19:52:57.781423 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.781389 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j"] Jul 08 19:52:57.784672 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.784653 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:57.807787 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.807763 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:57.809133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.809113 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j"] Jul 08 19:52:57.908112 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.908028 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps9fc\" (UniqueName: \"kubernetes.io/projected/90f37c07-46cd-4185-b4e7-46a1491826e7-kube-api-access-ps9fc\") pod \"90f37c07-46cd-4185-b4e7-46a1491826e7\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " Jul 08 19:52:57.908112 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.908063 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90f37c07-46cd-4185-b4e7-46a1491826e7-extensions-socket-volume\") pod \"90f37c07-46cd-4185-b4e7-46a1491826e7\" (UID: \"90f37c07-46cd-4185-b4e7-46a1491826e7\") " Jul 08 19:52:57.908324 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.908265 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdjw2\" (UniqueName: \"kubernetes.io/projected/d762646a-1485-4cea-975d-af66d3a7283b-kube-api-access-gdjw2\") pod \"kuadrant-operator-controller-manager-86878c769f-8dr6j\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:57.908324 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.908312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d762646a-1485-4cea-975d-af66d3a7283b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8dr6j\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:57.908562 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.908535 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f37c07-46cd-4185-b4e7-46a1491826e7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "90f37c07-46cd-4185-b4e7-46a1491826e7" (UID: "90f37c07-46cd-4185-b4e7-46a1491826e7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:57.910303 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:57.910283 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90f37c07-46cd-4185-b4e7-46a1491826e7-kube-api-access-ps9fc" (OuterVolumeSpecName: "kube-api-access-ps9fc") pod "90f37c07-46cd-4185-b4e7-46a1491826e7" (UID: "90f37c07-46cd-4185-b4e7-46a1491826e7"). InnerVolumeSpecName "kube-api-access-ps9fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:52:58.009014 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.008978 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdjw2\" (UniqueName: \"kubernetes.io/projected/d762646a-1485-4cea-975d-af66d3a7283b-kube-api-access-gdjw2\") pod \"kuadrant-operator-controller-manager-86878c769f-8dr6j\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:58.009241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.009023 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d762646a-1485-4cea-975d-af66d3a7283b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8dr6j\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:58.009241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.009098 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps9fc\" (UniqueName: \"kubernetes.io/projected/90f37c07-46cd-4185-b4e7-46a1491826e7-kube-api-access-ps9fc\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:52:58.009241 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.009108 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90f37c07-46cd-4185-b4e7-46a1491826e7-extensions-socket-volume\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:52:58.009519 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.009495 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d762646a-1485-4cea-975d-af66d3a7283b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8dr6j\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:58.018441 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.018415 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdjw2\" (UniqueName: \"kubernetes.io/projected/d762646a-1485-4cea-975d-af66d3a7283b-kube-api-access-gdjw2\") pod \"kuadrant-operator-controller-manager-86878c769f-8dr6j\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:58.094322 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.094289 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:58.240502 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.240475 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j"] Jul 08 19:52:58.242275 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:52:58.242246 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd762646a_1485_4cea_975d_af66d3a7283b.slice/crio-ee5d9b5e785e3d465fe6a6d876d44d5f19492280710582c8a062b02bbad3372d WatchSource:0}: Error finding container ee5d9b5e785e3d465fe6a6d876d44d5f19492280710582c8a062b02bbad3372d: Status 404 returned error can't find the container with id ee5d9b5e785e3d465fe6a6d876d44d5f19492280710582c8a062b02bbad3372d Jul 08 19:52:58.329837 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.329804 2569 generic.go:356] "Generic (PLEG): container finished" podID="90f37c07-46cd-4185-b4e7-46a1491826e7" containerID="1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc" exitCode=0 Jul 08 19:52:58.330182 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.329852 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" Jul 08 19:52:58.330182 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.329893 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" event={"ID":"90f37c07-46cd-4185-b4e7-46a1491826e7","Type":"ContainerDied","Data":"1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc"} Jul 08 19:52:58.330182 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.329944 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st" event={"ID":"90f37c07-46cd-4185-b4e7-46a1491826e7","Type":"ContainerDied","Data":"30b746868e3193697afe0da547e16cfa8448c51f225951f85f534983659fc946"} Jul 08 19:52:58.330182 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.329975 2569 scope.go:117] "RemoveContainer" containerID="1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc" Jul 08 19:52:58.331728 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.331697 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" event={"ID":"d762646a-1485-4cea-975d-af66d3a7283b","Type":"ContainerStarted","Data":"b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37"} Jul 08 19:52:58.331863 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.331754 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" event={"ID":"d762646a-1485-4cea-975d-af66d3a7283b","Type":"ContainerStarted","Data":"ee5d9b5e785e3d465fe6a6d876d44d5f19492280710582c8a062b02bbad3372d"} Jul 08 19:52:58.331863 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.331818 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:52:58.338387 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.338367 2569 scope.go:117] "RemoveContainer" containerID="1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc" Jul 08 19:52:58.338671 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:52:58.338642 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc\": container with ID starting with 1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc not found: ID does not exist" containerID="1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc" Jul 08 19:52:58.338751 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.338680 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc"} err="failed to get container status \"1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc\": rpc error: code = NotFound desc = could not find container \"1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc\": container with ID starting with 1d43b132193e18950975ae267cc8a86a592850d2b383d532142af45b03b4ccfc not found: ID does not exist" Jul 08 19:52:58.374910 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.374882 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st"] Jul 08 19:52:58.379537 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.379506 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fz6st"] Jul 08 19:52:58.857429 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:52:58.857392 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90f37c07-46cd-4185-b4e7-46a1491826e7" path="/var/lib/kubelet/pods/90f37c07-46cd-4185-b4e7-46a1491826e7/volumes" Jul 08 19:53:09.339783 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:09.339746 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 19:53:09.367214 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:09.367161 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" podStartSLOduration=12.367142354 podStartE2EDuration="12.367142354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:58.438055685 +0000 UTC m=+674.095163179" watchObservedRunningTime="2026-07-08 19:53:09.367142354 +0000 UTC m=+685.024249834" Jul 08 19:53:14.051624 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.051586 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl"] Jul 08 19:53:14.052103 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.051923 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f37c07-46cd-4185-b4e7-46a1491826e7" containerName="manager" Jul 08 19:53:14.052103 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.051935 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f37c07-46cd-4185-b4e7-46a1491826e7" containerName="manager" Jul 08 19:53:14.052103 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.051992 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f37c07-46cd-4185-b4e7-46a1491826e7" containerName="manager" Jul 08 19:53:14.065564 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.065538 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.070288 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.070253 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl"] Jul 08 19:53:14.070753 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.070731 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fq6hb\"" Jul 08 19:53:14.132947 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.132906 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.132947 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.132950 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133178 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.132978 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133178 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.132999 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133178 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.133030 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133178 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.133077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133178 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.133101 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133178 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.133153 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws7qz\" (UniqueName: \"kubernetes.io/projected/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-kube-api-access-ws7qz\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.133419 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.133235 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.233830 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233791 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.233830 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233834 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws7qz\" (UniqueName: \"kubernetes.io/projected/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-kube-api-access-ws7qz\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233882 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233901 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233925 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233966 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.233987 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234082 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234046 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234491 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234440 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234555 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234501 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234555 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234532 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234657 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234560 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.234967 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.234938 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.236353 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.236333 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.236489 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.236471 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.243193 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.243173 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.243470 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.243441 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws7qz\" (UniqueName: \"kubernetes.io/projected/ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b-kube-api-access-ws7qz\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ftnnl\" (UID: \"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.377987 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.377942 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:14.513994 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.513967 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl"] Jul 08 19:53:14.515759 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:53:14.515726 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc6cea3_48a3_4f5f_87d5_ac7e7f97c74b.slice/crio-83c8c422a72586874cd1fd8aea24e61e4e08a77dea5915d72960a0393c7471f1 WatchSource:0}: Error finding container 83c8c422a72586874cd1fd8aea24e61e4e08a77dea5915d72960a0393c7471f1: Status 404 returned error can't find the container with id 83c8c422a72586874cd1fd8aea24e61e4e08a77dea5915d72960a0393c7471f1 Jul 08 19:53:14.517909 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.517879 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:53:14.518036 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.517979 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:53:14.518036 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:14.518025 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:53:15.410622 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:15.410588 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" event={"ID":"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b","Type":"ContainerStarted","Data":"e4295148260e9eebc6c57555666c45b7c1e037cd0231b44c2cfec252f0eed4e9"} Jul 08 19:53:15.411007 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:15.410648 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" event={"ID":"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b","Type":"ContainerStarted","Data":"83c8c422a72586874cd1fd8aea24e61e4e08a77dea5915d72960a0393c7471f1"} Jul 08 19:53:15.435953 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:15.435901 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" podStartSLOduration=1.435884031 podStartE2EDuration="1.435884031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:53:15.434687726 +0000 UTC m=+691.091795206" watchObservedRunningTime="2026-07-08 19:53:15.435884031 +0000 UTC m=+691.092991511" Jul 08 19:53:16.379056 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:16.379018 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:16.383570 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:16.383544 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:16.415350 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:16.415317 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:16.416297 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:16.416283 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:53:32.737568 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:32.737536 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-jttbt"] Jul 08 19:53:32.778432 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:32.778393 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jttbt"] Jul 08 19:53:32.778609 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:32.778493 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:32.781471 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:32.781451 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n4hsz\"" Jul 08 19:53:32.902161 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:32.902120 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dcr9\" (UniqueName: \"kubernetes.io/projected/d61edbb6-1557-4d2a-adfa-19af361ad640-kube-api-access-6dcr9\") pod \"authorino-58cd955675-jttbt\" (UID: \"d61edbb6-1557-4d2a-adfa-19af361ad640\") " pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:33.002880 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:33.002794 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dcr9\" (UniqueName: \"kubernetes.io/projected/d61edbb6-1557-4d2a-adfa-19af361ad640-kube-api-access-6dcr9\") pod \"authorino-58cd955675-jttbt\" (UID: \"d61edbb6-1557-4d2a-adfa-19af361ad640\") " pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:33.011170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:33.011142 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dcr9\" (UniqueName: \"kubernetes.io/projected/d61edbb6-1557-4d2a-adfa-19af361ad640-kube-api-access-6dcr9\") pod \"authorino-58cd955675-jttbt\" (UID: \"d61edbb6-1557-4d2a-adfa-19af361ad640\") " pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:33.088159 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:33.088109 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:33.218507 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:33.218480 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jttbt"] Jul 08 19:53:33.221054 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:53:33.221026 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd61edbb6_1557_4d2a_adfa_19af361ad640.slice/crio-62a9f64017df30f81ce5e7593417a83c1334d8ae99ed273b1b0daa245352be0f WatchSource:0}: Error finding container 62a9f64017df30f81ce5e7593417a83c1334d8ae99ed273b1b0daa245352be0f: Status 404 returned error can't find the container with id 62a9f64017df30f81ce5e7593417a83c1334d8ae99ed273b1b0daa245352be0f Jul 08 19:53:33.488253 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:33.488217 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jttbt" event={"ID":"d61edbb6-1557-4d2a-adfa-19af361ad640","Type":"ContainerStarted","Data":"62a9f64017df30f81ce5e7593417a83c1334d8ae99ed273b1b0daa245352be0f"} Jul 08 19:53:36.504321 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:36.504284 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jttbt" event={"ID":"d61edbb6-1557-4d2a-adfa-19af361ad640","Type":"ContainerStarted","Data":"7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790"} Jul 08 19:53:36.525978 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:36.525930 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-jttbt" podStartSLOduration=1.738033648 podStartE2EDuration="4.52591382s" totalImagesPullingTime="2.787880172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:53:36.525420057 +0000 UTC m=+712.182527538" watchObservedRunningTime="2026-07-08 19:53:36.52591382 +0000 UTC m=+712.183021299" Jul 08 19:53:37.682332 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:37.682254 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jttbt"] Jul 08 19:53:38.513632 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:38.513588 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-jttbt" podUID="d61edbb6-1557-4d2a-adfa-19af361ad640" containerName="authorino" containerID="cri-o://7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790" gracePeriod=30 Jul 08 19:53:38.757294 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:38.757268 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:38.852644 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:38.852602 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dcr9\" (UniqueName: \"kubernetes.io/projected/d61edbb6-1557-4d2a-adfa-19af361ad640-kube-api-access-6dcr9\") pod \"d61edbb6-1557-4d2a-adfa-19af361ad640\" (UID: \"d61edbb6-1557-4d2a-adfa-19af361ad640\") " Jul 08 19:53:38.854782 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:38.854757 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d61edbb6-1557-4d2a-adfa-19af361ad640-kube-api-access-6dcr9" (OuterVolumeSpecName: "kube-api-access-6dcr9") pod "d61edbb6-1557-4d2a-adfa-19af361ad640" (UID: "d61edbb6-1557-4d2a-adfa-19af361ad640"). InnerVolumeSpecName "kube-api-access-6dcr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:38.954007 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:38.953970 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dcr9\" (UniqueName: \"kubernetes.io/projected/d61edbb6-1557-4d2a-adfa-19af361ad640-kube-api-access-6dcr9\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:53:39.519097 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.519065 2569 generic.go:356] "Generic (PLEG): container finished" podID="d61edbb6-1557-4d2a-adfa-19af361ad640" containerID="7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790" exitCode=0 Jul 08 19:53:39.519300 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.519111 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jttbt" Jul 08 19:53:39.519300 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.519158 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jttbt" event={"ID":"d61edbb6-1557-4d2a-adfa-19af361ad640","Type":"ContainerDied","Data":"7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790"} Jul 08 19:53:39.519300 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.519221 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jttbt" event={"ID":"d61edbb6-1557-4d2a-adfa-19af361ad640","Type":"ContainerDied","Data":"62a9f64017df30f81ce5e7593417a83c1334d8ae99ed273b1b0daa245352be0f"} Jul 08 19:53:39.519300 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.519222 2569 scope.go:117] "RemoveContainer" containerID="7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790" Jul 08 19:53:39.527180 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.527156 2569 scope.go:117] "RemoveContainer" containerID="7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790" Jul 08 19:53:39.527485 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:53:39.527467 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790\": container with ID starting with 7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790 not found: ID does not exist" containerID="7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790" Jul 08 19:53:39.527536 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.527493 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790"} err="failed to get container status \"7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790\": rpc error: code = NotFound desc = could not find container \"7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790\": container with ID starting with 7d8a68844b051b5cf17ddffddd167b4576aebb02f3dc9e1607952c024a795790 not found: ID does not exist" Jul 08 19:53:39.541181 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.541154 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jttbt"] Jul 08 19:53:39.546742 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:39.546720 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-jttbt"] Jul 08 19:53:40.857483 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:53:40.857453 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d61edbb6-1557-4d2a-adfa-19af361ad640" path="/var/lib/kubelet/pods/d61edbb6-1557-4d2a-adfa-19af361ad640/volumes" Jul 08 19:54:06.525696 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.525659 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxfss"] Jul 08 19:54:06.526170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.526052 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d61edbb6-1557-4d2a-adfa-19af361ad640" containerName="authorino" Jul 08 19:54:06.526170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.526068 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d61edbb6-1557-4d2a-adfa-19af361ad640" containerName="authorino" Jul 08 19:54:06.526170 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.526134 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d61edbb6-1557-4d2a-adfa-19af361ad640" containerName="authorino" Jul 08 19:54:06.529698 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.529680 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:06.533873 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.533839 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n4hsz\"" Jul 08 19:54:06.535776 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.535750 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxfss"] Jul 08 19:54:06.582801 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.582755 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cprmn\" (UniqueName: \"kubernetes.io/projected/d4388899-9ac1-4ffb-be83-89b283ee02f5-kube-api-access-cprmn\") pod \"authorino-5d4b67c446-sxfss\" (UID: \"d4388899-9ac1-4ffb-be83-89b283ee02f5\") " pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:06.684112 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.684077 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprmn\" (UniqueName: \"kubernetes.io/projected/d4388899-9ac1-4ffb-be83-89b283ee02f5-kube-api-access-cprmn\") pod \"authorino-5d4b67c446-sxfss\" (UID: \"d4388899-9ac1-4ffb-be83-89b283ee02f5\") " pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:06.694602 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.694568 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cprmn\" (UniqueName: \"kubernetes.io/projected/d4388899-9ac1-4ffb-be83-89b283ee02f5-kube-api-access-cprmn\") pod \"authorino-5d4b67c446-sxfss\" (UID: \"d4388899-9ac1-4ffb-be83-89b283ee02f5\") " pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:06.756864 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.756830 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxfss"] Jul 08 19:54:06.757092 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.757080 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:06.784971 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.784894 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74bf8c6c99-cjvdp"] Jul 08 19:54:06.787445 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.787423 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:06.802983 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.802957 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74bf8c6c99-cjvdp"] Jul 08 19:54:06.884913 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.884883 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbrhk\" (UniqueName: \"kubernetes.io/projected/c57cc517-4968-4f7c-ab30-58682c64eddb-kube-api-access-fbrhk\") pod \"authorino-74bf8c6c99-cjvdp\" (UID: \"c57cc517-4968-4f7c-ab30-58682c64eddb\") " pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:06.897769 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.897747 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxfss"] Jul 08 19:54:06.899856 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:54:06.899828 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4388899_9ac1_4ffb_be83_89b283ee02f5.slice/crio-da0997e3fb736e13c9d51b30ee6eba4f87044caebd3237e33bdabb1bad4aff51 WatchSource:0}: Error finding container da0997e3fb736e13c9d51b30ee6eba4f87044caebd3237e33bdabb1bad4aff51: Status 404 returned error can't find the container with id da0997e3fb736e13c9d51b30ee6eba4f87044caebd3237e33bdabb1bad4aff51 Jul 08 19:54:06.985650 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.985615 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbrhk\" (UniqueName: \"kubernetes.io/projected/c57cc517-4968-4f7c-ab30-58682c64eddb-kube-api-access-fbrhk\") pod \"authorino-74bf8c6c99-cjvdp\" (UID: \"c57cc517-4968-4f7c-ab30-58682c64eddb\") " pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:06.996232 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:06.996173 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbrhk\" (UniqueName: \"kubernetes.io/projected/c57cc517-4968-4f7c-ab30-58682c64eddb-kube-api-access-fbrhk\") pod \"authorino-74bf8c6c99-cjvdp\" (UID: \"c57cc517-4968-4f7c-ab30-58682c64eddb\") " pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:07.057697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.057664 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bf8c6c99-cjvdp"] Jul 08 19:54:07.057909 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.057898 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:07.195091 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.195064 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bf8c6c99-cjvdp"] Jul 08 19:54:07.197747 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:54:07.197721 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc57cc517_4968_4f7c_ab30_58682c64eddb.slice/crio-33cc93fce0e8fdb78c9f5e2af790259fb564c08dd91a55d4b7efb891bf25a000 WatchSource:0}: Error finding container 33cc93fce0e8fdb78c9f5e2af790259fb564c08dd91a55d4b7efb891bf25a000: Status 404 returned error can't find the container with id 33cc93fce0e8fdb78c9f5e2af790259fb564c08dd91a55d4b7efb891bf25a000 Jul 08 19:54:07.648153 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.648117 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" event={"ID":"c57cc517-4968-4f7c-ab30-58682c64eddb","Type":"ContainerStarted","Data":"54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d"} Jul 08 19:54:07.648623 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.648163 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" event={"ID":"c57cc517-4968-4f7c-ab30-58682c64eddb","Type":"ContainerStarted","Data":"33cc93fce0e8fdb78c9f5e2af790259fb564c08dd91a55d4b7efb891bf25a000"} Jul 08 19:54:07.648623 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.648190 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" podUID="c57cc517-4968-4f7c-ab30-58682c64eddb" containerName="authorino" containerID="cri-o://54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d" gracePeriod=30 Jul 08 19:54:07.650096 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.650054 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sxfss" event={"ID":"d4388899-9ac1-4ffb-be83-89b283ee02f5","Type":"ContainerStarted","Data":"005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb"} Jul 08 19:54:07.650096 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.650097 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sxfss" event={"ID":"d4388899-9ac1-4ffb-be83-89b283ee02f5","Type":"ContainerStarted","Data":"da0997e3fb736e13c9d51b30ee6eba4f87044caebd3237e33bdabb1bad4aff51"} Jul 08 19:54:07.650335 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.650133 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-sxfss" podUID="d4388899-9ac1-4ffb-be83-89b283ee02f5" containerName="authorino" containerID="cri-o://005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb" gracePeriod=30 Jul 08 19:54:07.687560 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.687511 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" podStartSLOduration=1.3419946010000001 podStartE2EDuration="1.687494708s" totalImagesPullingTime="345.500107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:07.685142832 +0000 UTC m=+743.342250312" watchObservedRunningTime="2026-07-08 19:54:07.687494708 +0000 UTC m=+743.344602188" Jul 08 19:54:07.714080 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.714027 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-sxfss" podStartSLOduration=1.096222446 podStartE2EDuration="1.714011327s" totalImagesPullingTime="617.788881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:07.712863935 +0000 UTC m=+743.369971416" watchObservedRunningTime="2026-07-08 19:54:07.714011327 +0000 UTC m=+743.371118806" Jul 08 19:54:07.919076 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.919047 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:07.928805 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.928784 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:07.992068 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.992040 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbrhk\" (UniqueName: \"kubernetes.io/projected/c57cc517-4968-4f7c-ab30-58682c64eddb-kube-api-access-fbrhk\") pod \"c57cc517-4968-4f7c-ab30-58682c64eddb\" (UID: \"c57cc517-4968-4f7c-ab30-58682c64eddb\") " Jul 08 19:54:07.992228 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.992120 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cprmn\" (UniqueName: \"kubernetes.io/projected/d4388899-9ac1-4ffb-be83-89b283ee02f5-kube-api-access-cprmn\") pod \"d4388899-9ac1-4ffb-be83-89b283ee02f5\" (UID: \"d4388899-9ac1-4ffb-be83-89b283ee02f5\") " Jul 08 19:54:07.994213 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.994172 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4388899-9ac1-4ffb-be83-89b283ee02f5-kube-api-access-cprmn" (OuterVolumeSpecName: "kube-api-access-cprmn") pod "d4388899-9ac1-4ffb-be83-89b283ee02f5" (UID: "d4388899-9ac1-4ffb-be83-89b283ee02f5"). InnerVolumeSpecName "kube-api-access-cprmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:54:07.994337 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:07.994217 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57cc517-4968-4f7c-ab30-58682c64eddb-kube-api-access-fbrhk" (OuterVolumeSpecName: "kube-api-access-fbrhk") pod "c57cc517-4968-4f7c-ab30-58682c64eddb" (UID: "c57cc517-4968-4f7c-ab30-58682c64eddb"). InnerVolumeSpecName "kube-api-access-fbrhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:54:08.092700 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.092587 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbrhk\" (UniqueName: \"kubernetes.io/projected/c57cc517-4968-4f7c-ab30-58682c64eddb-kube-api-access-fbrhk\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:54:08.092700 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.092632 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cprmn\" (UniqueName: \"kubernetes.io/projected/d4388899-9ac1-4ffb-be83-89b283ee02f5-kube-api-access-cprmn\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 19:54:08.655676 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.655643 2569 generic.go:356] "Generic (PLEG): container finished" podID="c57cc517-4968-4f7c-ab30-58682c64eddb" containerID="54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d" exitCode=0 Jul 08 19:54:08.656119 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.655689 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" Jul 08 19:54:08.656119 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.655714 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" event={"ID":"c57cc517-4968-4f7c-ab30-58682c64eddb","Type":"ContainerDied","Data":"54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d"} Jul 08 19:54:08.656119 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.655749 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bf8c6c99-cjvdp" event={"ID":"c57cc517-4968-4f7c-ab30-58682c64eddb","Type":"ContainerDied","Data":"33cc93fce0e8fdb78c9f5e2af790259fb564c08dd91a55d4b7efb891bf25a000"} Jul 08 19:54:08.656119 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.655771 2569 scope.go:117] "RemoveContainer" containerID="54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d" Jul 08 19:54:08.657079 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.657058 2569 generic.go:356] "Generic (PLEG): container finished" podID="d4388899-9ac1-4ffb-be83-89b283ee02f5" containerID="005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb" exitCode=0 Jul 08 19:54:08.657188 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.657107 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxfss" Jul 08 19:54:08.657188 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.657121 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sxfss" event={"ID":"d4388899-9ac1-4ffb-be83-89b283ee02f5","Type":"ContainerDied","Data":"005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb"} Jul 08 19:54:08.657188 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.657149 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sxfss" event={"ID":"d4388899-9ac1-4ffb-be83-89b283ee02f5","Type":"ContainerDied","Data":"da0997e3fb736e13c9d51b30ee6eba4f87044caebd3237e33bdabb1bad4aff51"} Jul 08 19:54:08.665752 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.665735 2569 scope.go:117] "RemoveContainer" containerID="54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d" Jul 08 19:54:08.666028 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:54:08.666009 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d\": container with ID starting with 54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d not found: ID does not exist" containerID="54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d" Jul 08 19:54:08.666115 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.666035 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d"} err="failed to get container status \"54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d\": rpc error: code = NotFound desc = could not find container \"54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d\": container with ID starting with 54ea2c17f9f2f527794c066265fa7ca42a8911940c546840ddae9f3b714ee19d not found: ID does not exist" Jul 08 19:54:08.666115 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.666052 2569 scope.go:117] "RemoveContainer" containerID="005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb" Jul 08 19:54:08.673501 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.673486 2569 scope.go:117] "RemoveContainer" containerID="005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb" Jul 08 19:54:08.673720 ip-10-0-133-133 kubenswrapper[2569]: E0708 19:54:08.673702 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb\": container with ID starting with 005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb not found: ID does not exist" containerID="005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb" Jul 08 19:54:08.673773 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.673727 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb"} err="failed to get container status \"005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb\": rpc error: code = NotFound desc = could not find container \"005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb\": container with ID starting with 005e7e7bc3aff0b548406988c12d151f3c3630e4bc6fbda31497dae331b3a8cb not found: ID does not exist" Jul 08 19:54:08.682032 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.682011 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bf8c6c99-cjvdp"] Jul 08 19:54:08.684121 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.684099 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74bf8c6c99-cjvdp"] Jul 08 19:54:08.698867 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.698840 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxfss"] Jul 08 19:54:08.706718 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.706691 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxfss"] Jul 08 19:54:08.857976 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.857942 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c57cc517-4968-4f7c-ab30-58682c64eddb" path="/var/lib/kubelet/pods/c57cc517-4968-4f7c-ab30-58682c64eddb/volumes" Jul 08 19:54:08.858344 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:08.858330 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4388899-9ac1-4ffb-be83-89b283ee02f5" path="/var/lib/kubelet/pods/d4388899-9ac1-4ffb-be83-89b283ee02f5/volumes" Jul 08 19:54:58.883003 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:58.882968 2569 generic.go:356] "Generic (PLEG): container finished" podID="ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b" containerID="e4295148260e9eebc6c57555666c45b7c1e037cd0231b44c2cfec252f0eed4e9" exitCode=0 Jul 08 19:54:58.883450 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:58.883042 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" event={"ID":"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b","Type":"ContainerDied","Data":"e4295148260e9eebc6c57555666c45b7c1e037cd0231b44c2cfec252f0eed4e9"} Jul 08 19:54:58.883503 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:58.883489 2569 scope.go:117] "RemoveContainer" containerID="e4295148260e9eebc6c57555666c45b7c1e037cd0231b44c2cfec252f0eed4e9" Jul 08 19:54:58.884452 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:58.884425 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:54:58.884522 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:58.884501 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:54:58.884562 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:58.884547 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 19:54:59.890059 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:54:59.890023 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" event={"ID":"ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b","Type":"ContainerStarted","Data":"8506fe32dcf81466a1a945bf18027d7161718e80eb5e71425adcac5f9a971d15"} Jul 08 19:55:00.378522 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:00.378489 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:55:00.379792 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:00.379763 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ftnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:55:00.379933 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:00.379829 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" podUID="ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 08 19:55:01.379173 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:01.379138 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ftnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:55:01.379661 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:01.379191 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" podUID="ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 08 19:55:01.415697 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:01.415668 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:55:02.378724 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:02.378689 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ftnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:55:02.378902 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:02.378744 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" podUID="ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 08 19:55:03.378550 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:03.378510 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ftnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:55:03.378938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:03.378568 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" podUID="ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 08 19:55:04.382955 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:04.382924 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:55:04.384075 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:04.384057 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ftnnl" Jul 08 19:55:17.409874 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.409797 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6"] Jul 08 19:55:17.410248 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.410124 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4388899-9ac1-4ffb-be83-89b283ee02f5" containerName="authorino" Jul 08 19:55:17.410248 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.410136 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4388899-9ac1-4ffb-be83-89b283ee02f5" containerName="authorino" Jul 08 19:55:17.410248 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.410148 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57cc517-4968-4f7c-ab30-58682c64eddb" containerName="authorino" Jul 08 19:55:17.410248 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.410154 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57cc517-4968-4f7c-ab30-58682c64eddb" containerName="authorino" Jul 08 19:55:17.410248 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.410228 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57cc517-4968-4f7c-ab30-58682c64eddb" containerName="authorino" Jul 08 19:55:17.410248 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.410237 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4388899-9ac1-4ffb-be83-89b283ee02f5" containerName="authorino" Jul 08 19:55:17.413079 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.413058 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.417957 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.417930 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sqmkz\"" Jul 08 19:55:17.417957 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.417946 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 19:55:17.418147 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.417988 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 19:55:17.418147 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.418070 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 19:55:17.428806 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.428783 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6"] Jul 08 19:55:17.471974 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.471940 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52e0c37-1875-4659-93e1-35ab4ea1b355-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.472133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.471985 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.472133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.472030 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.472133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.472066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqb26\" (UniqueName: \"kubernetes.io/projected/d52e0c37-1875-4659-93e1-35ab4ea1b355-kube-api-access-nqb26\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.472133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.472087 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.472133 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.472124 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573010 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.572973 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqb26\" (UniqueName: \"kubernetes.io/projected/d52e0c37-1875-4659-93e1-35ab4ea1b355-kube-api-access-nqb26\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573010 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573013 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573256 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573050 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573256 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573095 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52e0c37-1875-4659-93e1-35ab4ea1b355-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573256 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573129 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573256 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573168 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573560 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573533 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573560 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573557 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.573692 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.573574 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.575443 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.575414 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52e0c37-1875-4659-93e1-35ab4ea1b355-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.575659 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.575643 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52e0c37-1875-4659-93e1-35ab4ea1b355-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.582660 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.582635 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqb26\" (UniqueName: \"kubernetes.io/projected/d52e0c37-1875-4659-93e1-35ab4ea1b355-kube-api-access-nqb26\") pod \"e2e-distinct-simulated-kserve-59975d69f9-szdf6\" (UID: \"d52e0c37-1875-4659-93e1-35ab4ea1b355\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.723649 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.723559 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:17.863918 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.863894 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6"] Jul 08 19:55:17.865882 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:55:17.865845 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52e0c37_1875_4659_93e1_35ab4ea1b355.slice/crio-f4cb64618b4cdf3ae6ddb9a51ec842ba8536895959aae70d3450cd4fb649903c WatchSource:0}: Error finding container f4cb64618b4cdf3ae6ddb9a51ec842ba8536895959aae70d3450cd4fb649903c: Status 404 returned error can't find the container with id f4cb64618b4cdf3ae6ddb9a51ec842ba8536895959aae70d3450cd4fb649903c Jul 08 19:55:17.967578 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:17.967542 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" event={"ID":"d52e0c37-1875-4659-93e1-35ab4ea1b355","Type":"ContainerStarted","Data":"f4cb64618b4cdf3ae6ddb9a51ec842ba8536895959aae70d3450cd4fb649903c"} Jul 08 19:55:19.981704 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:19.981660 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" event={"ID":"d52e0c37-1875-4659-93e1-35ab4ea1b355","Type":"ContainerStarted","Data":"10fea73d81754cc39ed676792dbd4bd7adf0b75df17b0b5a3b6f9be63b8cbf80"} Jul 08 19:55:20.003065 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:20.003006 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" podStartSLOduration=1.962884764 podStartE2EDuration="3.002980832s" totalImagesPullingTime="1.040096068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:55:20.000943131 +0000 UTC m=+815.658050645" watchObservedRunningTime="2026-07-08 19:55:20.002980832 +0000 UTC m=+815.660088314" Jul 08 19:55:27.724261 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:27.724184 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:27.724893 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:27.724292 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:27.736940 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:27.736908 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:28.029489 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:28.029398 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-szdf6" Jul 08 19:55:45.796475 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.796437 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk"] Jul 08 19:55:45.799879 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.799855 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:45.804849 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.804826 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 19:55:45.810573 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.810548 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk"] Jul 08 19:55:45.915726 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.915692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:45.915938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.915749 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:45.915938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.915772 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01a7935f-5877-4bf3-aef8-fff086421fda-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:45.915938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.915797 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc5g9\" (UniqueName: \"kubernetes.io/projected/01a7935f-5877-4bf3-aef8-fff086421fda-kube-api-access-cc5g9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:45.915938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.915835 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:45.915938 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:45.915886 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.016992 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.016960 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc5g9\" (UniqueName: \"kubernetes.io/projected/01a7935f-5877-4bf3-aef8-fff086421fda-kube-api-access-cc5g9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017183 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017004 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017183 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017037 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017183 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017070 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017183 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017110 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017183 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01a7935f-5877-4bf3-aef8-fff086421fda-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017483 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017431 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017535 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017495 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.017579 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.017549 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.019390 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.019371 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01a7935f-5877-4bf3-aef8-fff086421fda-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.019590 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.019570 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01a7935f-5877-4bf3-aef8-fff086421fda-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.025435 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.025412 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc5g9\" (UniqueName: \"kubernetes.io/projected/01a7935f-5877-4bf3-aef8-fff086421fda-kube-api-access-cc5g9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-plxkk\" (UID: \"01a7935f-5877-4bf3-aef8-fff086421fda\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.111151 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.111077 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:46.250019 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:46.249992 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk"] Jul 08 19:55:46.252372 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:55:46.252340 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a7935f_5877_4bf3_aef8_fff086421fda.slice/crio-f715cc584ae06ed09934bd9808cab1b1ab90a64e8f10035b4e018e010c66f6d3 WatchSource:0}: Error finding container f715cc584ae06ed09934bd9808cab1b1ab90a64e8f10035b4e018e010c66f6d3: Status 404 returned error can't find the container with id f715cc584ae06ed09934bd9808cab1b1ab90a64e8f10035b4e018e010c66f6d3 Jul 08 19:55:47.105052 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:47.105011 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" event={"ID":"01a7935f-5877-4bf3-aef8-fff086421fda","Type":"ContainerStarted","Data":"a8809bde8a3fa5446fba605ebfd803cde55a01155a20bf92e447745f874b079e"} Jul 08 19:55:47.105459 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:47.105074 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" event={"ID":"01a7935f-5877-4bf3-aef8-fff086421fda","Type":"ContainerStarted","Data":"f715cc584ae06ed09934bd9808cab1b1ab90a64e8f10035b4e018e010c66f6d3"} Jul 08 19:55:47.125325 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:47.125273 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" podStartSLOduration=1.939377175 podStartE2EDuration="2.125254243s" totalImagesPullingTime="185.877068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:55:47.124798496 +0000 UTC m=+842.781905976" watchObservedRunningTime="2026-07-08 19:55:47.125254243 +0000 UTC m=+842.782361724" Jul 08 19:55:56.111895 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:56.111854 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:56.112318 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:56.111916 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:56.124182 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:56.124158 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:55:56.158440 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:55:56.158403 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-plxkk" Jul 08 19:57:11.036237 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.036183 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7469bcd-692jn"] Jul 08 19:57:11.039742 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.039718 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.044073 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.044053 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n4hsz\"" Jul 08 19:57:11.044180 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.044054 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 19:57:11.047837 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.047807 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7469bcd-692jn"] Jul 08 19:57:11.134278 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.134237 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9831de8-1949-4c88-bb0a-88585f8442c3-tls-cert\") pod \"authorino-57f7469bcd-692jn\" (UID: \"d9831de8-1949-4c88-bb0a-88585f8442c3\") " pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.134278 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.134283 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg2n2\" (UniqueName: \"kubernetes.io/projected/d9831de8-1949-4c88-bb0a-88585f8442c3-kube-api-access-jg2n2\") pod \"authorino-57f7469bcd-692jn\" (UID: \"d9831de8-1949-4c88-bb0a-88585f8442c3\") " pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.235403 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.235366 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9831de8-1949-4c88-bb0a-88585f8442c3-tls-cert\") pod \"authorino-57f7469bcd-692jn\" (UID: \"d9831de8-1949-4c88-bb0a-88585f8442c3\") " pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.235403 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.235405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg2n2\" (UniqueName: \"kubernetes.io/projected/d9831de8-1949-4c88-bb0a-88585f8442c3-kube-api-access-jg2n2\") pod \"authorino-57f7469bcd-692jn\" (UID: \"d9831de8-1949-4c88-bb0a-88585f8442c3\") " pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.237898 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.237869 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9831de8-1949-4c88-bb0a-88585f8442c3-tls-cert\") pod \"authorino-57f7469bcd-692jn\" (UID: \"d9831de8-1949-4c88-bb0a-88585f8442c3\") " pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.245593 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.245572 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg2n2\" (UniqueName: \"kubernetes.io/projected/d9831de8-1949-4c88-bb0a-88585f8442c3-kube-api-access-jg2n2\") pod \"authorino-57f7469bcd-692jn\" (UID: \"d9831de8-1949-4c88-bb0a-88585f8442c3\") " pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.349702 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.349670 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7469bcd-692jn" Jul 08 19:57:11.486735 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:11.486702 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7469bcd-692jn"] Jul 08 19:57:11.491035 ip-10-0-133-133 kubenswrapper[2569]: W0708 19:57:11.491011 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9831de8_1949_4c88_bb0a_88585f8442c3.slice/crio-9a167c409e330b5c714cf44b39514963c54a596d3a219de83844a47020cb8455 WatchSource:0}: Error finding container 9a167c409e330b5c714cf44b39514963c54a596d3a219de83844a47020cb8455: Status 404 returned error can't find the container with id 9a167c409e330b5c714cf44b39514963c54a596d3a219de83844a47020cb8455 Jul 08 19:57:12.494612 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:12.494576 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7469bcd-692jn" event={"ID":"d9831de8-1949-4c88-bb0a-88585f8442c3","Type":"ContainerStarted","Data":"e95fe298e3c0f74763cb9c516a6b093397aa5c2b8b96173d4b611222d2879c26"} Jul 08 19:57:12.495001 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:12.494638 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7469bcd-692jn" event={"ID":"d9831de8-1949-4c88-bb0a-88585f8442c3","Type":"ContainerStarted","Data":"9a167c409e330b5c714cf44b39514963c54a596d3a219de83844a47020cb8455"} Jul 08 19:57:12.544439 ip-10-0-133-133 kubenswrapper[2569]: I0708 19:57:12.544380 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7469bcd-692jn" podStartSLOduration=1.116707845 podStartE2EDuration="1.544362114s" totalImagesPullingTime="427.654269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:57:12.542807876 +0000 UTC m=+928.199915357" watchObservedRunningTime="2026-07-08 19:57:12.544362114 +0000 UTC m=+928.201469593" Jul 08 20:03:08.353653 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:08.353566 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j"] Jul 08 20:03:08.354187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:08.353801 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" podUID="d762646a-1485-4cea-975d-af66d3a7283b" containerName="manager" containerID="cri-o://b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37" gracePeriod=10 Jul 08 20:03:11.313643 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.313617 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 20:03:11.470135 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.470050 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdjw2\" (UniqueName: \"kubernetes.io/projected/d762646a-1485-4cea-975d-af66d3a7283b-kube-api-access-gdjw2\") pod \"d762646a-1485-4cea-975d-af66d3a7283b\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " Jul 08 20:03:11.470317 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.470159 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d762646a-1485-4cea-975d-af66d3a7283b-extensions-socket-volume\") pod \"d762646a-1485-4cea-975d-af66d3a7283b\" (UID: \"d762646a-1485-4cea-975d-af66d3a7283b\") " Jul 08 20:03:11.470541 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.470519 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d762646a-1485-4cea-975d-af66d3a7283b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d762646a-1485-4cea-975d-af66d3a7283b" (UID: "d762646a-1485-4cea-975d-af66d3a7283b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:03:11.472207 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.472177 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d762646a-1485-4cea-975d-af66d3a7283b-kube-api-access-gdjw2" (OuterVolumeSpecName: "kube-api-access-gdjw2") pod "d762646a-1485-4cea-975d-af66d3a7283b" (UID: "d762646a-1485-4cea-975d-af66d3a7283b"). InnerVolumeSpecName "kube-api-access-gdjw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:03:11.571742 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.571708 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdjw2\" (UniqueName: \"kubernetes.io/projected/d762646a-1485-4cea-975d-af66d3a7283b-kube-api-access-gdjw2\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:03:11.571742 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:11.571740 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d762646a-1485-4cea-975d-af66d3a7283b-extensions-socket-volume\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:03:12.152561 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.152526 2569 generic.go:356] "Generic (PLEG): container finished" podID="d762646a-1485-4cea-975d-af66d3a7283b" containerID="b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37" exitCode=0 Jul 08 20:03:12.152729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.152567 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" Jul 08 20:03:12.152729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.152613 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" event={"ID":"d762646a-1485-4cea-975d-af66d3a7283b","Type":"ContainerDied","Data":"b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37"} Jul 08 20:03:12.152729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.152660 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j" event={"ID":"d762646a-1485-4cea-975d-af66d3a7283b","Type":"ContainerDied","Data":"ee5d9b5e785e3d465fe6a6d876d44d5f19492280710582c8a062b02bbad3372d"} Jul 08 20:03:12.152729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.152681 2569 scope.go:117] "RemoveContainer" containerID="b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37" Jul 08 20:03:12.164028 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.163984 2569 scope.go:117] "RemoveContainer" containerID="b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37" Jul 08 20:03:12.164481 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:03:12.164457 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37\": container with ID starting with b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37 not found: ID does not exist" containerID="b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37" Jul 08 20:03:12.164586 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.164494 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37"} err="failed to get container status \"b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37\": rpc error: code = NotFound desc = could not find container \"b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37\": container with ID starting with b975e4d57569b3b86e0b57223bf1c9dc40cdf1c27eaebb84a81c2c6a5021fc37 not found: ID does not exist" Jul 08 20:03:12.177114 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.177086 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j"] Jul 08 20:03:12.181533 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.181511 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8dr6j"] Jul 08 20:03:12.858347 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:03:12.858305 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d762646a-1485-4cea-975d-af66d3a7283b" path="/var/lib/kubelet/pods/d762646a-1485-4cea-975d-af66d3a7283b/volumes" Jul 08 20:04:09.534770 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.534684 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7"] Jul 08 20:04:09.535222 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.535013 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d762646a-1485-4cea-975d-af66d3a7283b" containerName="manager" Jul 08 20:04:09.535222 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.535025 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d762646a-1485-4cea-975d-af66d3a7283b" containerName="manager" Jul 08 20:04:09.535222 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.535094 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d762646a-1485-4cea-975d-af66d3a7283b" containerName="manager" Jul 08 20:04:09.539152 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.539135 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.542112 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.542095 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gxwbh\"" Jul 08 20:04:09.549482 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.549458 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7"] Jul 08 20:04:09.674442 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.674387 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5e68922-80af-4d06-990f-820536c8e7d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnpg7\" (UID: \"e5e68922-80af-4d06-990f-820536c8e7d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.674641 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.674461 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8swbb\" (UniqueName: \"kubernetes.io/projected/e5e68922-80af-4d06-990f-820536c8e7d0-kube-api-access-8swbb\") pod \"kuadrant-operator-controller-manager-86878c769f-vnpg7\" (UID: \"e5e68922-80af-4d06-990f-820536c8e7d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.775933 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.775899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8swbb\" (UniqueName: \"kubernetes.io/projected/e5e68922-80af-4d06-990f-820536c8e7d0-kube-api-access-8swbb\") pod \"kuadrant-operator-controller-manager-86878c769f-vnpg7\" (UID: \"e5e68922-80af-4d06-990f-820536c8e7d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.776102 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.775989 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5e68922-80af-4d06-990f-820536c8e7d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnpg7\" (UID: \"e5e68922-80af-4d06-990f-820536c8e7d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.776354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.776338 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5e68922-80af-4d06-990f-820536c8e7d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnpg7\" (UID: \"e5e68922-80af-4d06-990f-820536c8e7d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.789456 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.789393 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8swbb\" (UniqueName: \"kubernetes.io/projected/e5e68922-80af-4d06-990f-820536c8e7d0-kube-api-access-8swbb\") pod \"kuadrant-operator-controller-manager-86878c769f-vnpg7\" (UID: \"e5e68922-80af-4d06-990f-820536c8e7d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.849524 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.849487 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:09.989923 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:09.989896 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7"] Jul 08 20:04:09.991860 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:04:09.991832 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e68922_80af_4d06_990f_820536c8e7d0.slice/crio-d28a7420c321124709ad68c23bfe0bc41f3def4699f14a19812abdee247afa59 WatchSource:0}: Error finding container d28a7420c321124709ad68c23bfe0bc41f3def4699f14a19812abdee247afa59: Status 404 returned error can't find the container with id d28a7420c321124709ad68c23bfe0bc41f3def4699f14a19812abdee247afa59 Jul 08 20:04:10.418212 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:10.418108 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" event={"ID":"e5e68922-80af-4d06-990f-820536c8e7d0","Type":"ContainerStarted","Data":"eba0f0fab16fd9ce5cbaede9ee872b50e88ac2eca2d668475b5c0175494e997e"} Jul 08 20:04:10.418212 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:10.418166 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" event={"ID":"e5e68922-80af-4d06-990f-820536c8e7d0","Type":"ContainerStarted","Data":"d28a7420c321124709ad68c23bfe0bc41f3def4699f14a19812abdee247afa59"} Jul 08 20:04:10.418408 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:10.418218 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:04:10.437748 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:10.437685 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" podStartSLOduration=1.437667389 podStartE2EDuration="1.437667389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:04:10.436104148 +0000 UTC m=+1346.093211629" watchObservedRunningTime="2026-07-08 20:04:10.437667389 +0000 UTC m=+1346.094774868" Jul 08 20:04:21.425286 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:04:21.425252 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnpg7" Jul 08 20:10:36.051480 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.051446 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9"] Jul 08 20:10:36.054749 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.054724 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.059445 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.059412 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5dd1d121-openshift-default-dockercfg-p9fhp\"" Jul 08 20:10:36.081572 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.081542 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9"] Jul 08 20:10:36.150239 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150186 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-socket\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150431 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150259 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld5wg\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-kube-api-access-ld5wg\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150431 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150320 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c17a6f-5643-4516-b712-a0f94ccfbd86-istiod-ca-cert\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150431 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150359 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-data\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150431 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150388 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-token\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150637 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150447 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-credential-socket\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150637 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150494 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-certs\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150637 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150524 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-podinfo\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.150637 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.150581 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-envoy\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251162 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251125 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld5wg\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-kube-api-access-ld5wg\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251178 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c17a6f-5643-4516-b712-a0f94ccfbd86-istiod-ca-cert\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251232 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-data\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-token\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251286 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-credential-socket\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251312 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-certs\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-podinfo\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251368 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-envoy\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251648 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251403 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-socket\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251821 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251792 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-socket\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251878 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251840 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-data\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.251938 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.251884 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-credential-socket\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.252080 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.252057 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c17a6f-5643-4516-b712-a0f94ccfbd86-istiod-ca-cert\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.252169 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.252060 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-certs\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.253822 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.253802 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-envoy\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.253946 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.253927 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-podinfo\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.262574 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.262553 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-token\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.262815 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.262798 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld5wg\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-kube-api-access-ld5wg\") pod \"e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.366069 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.366029 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:36.507816 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.507786 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9"] Jul 08 20:10:36.509757 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:10:36.509722 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c17a6f_5643_4516_b712_a0f94ccfbd86.slice/crio-6a173fa33ad0e7018d92ea43f727aa56825b28438fdbd883262b56543b668a91 WatchSource:0}: Error finding container 6a173fa33ad0e7018d92ea43f727aa56825b28438fdbd883262b56543b668a91: Status 404 returned error can't find the container with id 6a173fa33ad0e7018d92ea43f727aa56825b28438fdbd883262b56543b668a91 Jul 08 20:10:36.511861 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.511823 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:10:36.511997 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.511899 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:10:36.511997 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:36.511934 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:10:37.225986 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:37.225951 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" event={"ID":"93c17a6f-5643-4516-b712-a0f94ccfbd86","Type":"ContainerStarted","Data":"e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6"} Jul 08 20:10:37.226397 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:37.226008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" event={"ID":"93c17a6f-5643-4516-b712-a0f94ccfbd86","Type":"ContainerStarted","Data":"6a173fa33ad0e7018d92ea43f727aa56825b28438fdbd883262b56543b668a91"} Jul 08 20:10:37.260729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:37.260668 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" podStartSLOduration=1.260642743 podStartE2EDuration="1.260642743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:10:37.258187563 +0000 UTC m=+1732.915295044" watchObservedRunningTime="2026-07-08 20:10:37.260642743 +0000 UTC m=+1732.917750224" Jul 08 20:10:37.366522 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:37.366480 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:37.371436 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:37.371412 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:38.233873 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:38.233846 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:38.235092 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:38.235066 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:42.580128 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:42.580090 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9"] Jul 08 20:10:42.580628 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:42.580443 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" podUID="93c17a6f-5643-4516-b712-a0f94ccfbd86" containerName="istio-proxy" containerID="cri-o://e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6" gracePeriod=30 Jul 08 20:10:47.837439 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.837406 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:47.956174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956083 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-socket\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956139 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld5wg\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-kube-api-access-ld5wg\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956168 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-token\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956240 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-envoy\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956262 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-credential-socket\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956283 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-data\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956320 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c17a6f-5643-4516-b712-a0f94ccfbd86-istiod-ca-cert\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956361 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-certs\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956417 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-podinfo\") pod \"93c17a6f-5643-4516-b712-a0f94ccfbd86\" (UID: \"93c17a6f-5643-4516-b712-a0f94ccfbd86\") " Jul 08 20:10:47.956481 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956411 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:10:47.956817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956612 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:10:47.956817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956648 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:47.956817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956765 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:10:47.956817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956793 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c17a6f-5643-4516-b712-a0f94ccfbd86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:10:47.956969 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.956826 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-data" (OuterVolumeSpecName: "istio-data") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:10:47.958573 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.958537 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-kube-api-access-ld5wg" (OuterVolumeSpecName: "kube-api-access-ld5wg") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "kube-api-access-ld5wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:10:47.958700 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.958678 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:10:47.958844 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.958819 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:10:47.958927 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:47.958820 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-token" (OuterVolumeSpecName: "istio-token") pod "93c17a6f-5643-4516-b712-a0f94ccfbd86" (UID: "93c17a6f-5643-4516-b712-a0f94ccfbd86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:10:48.058037 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058000 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c17a6f-5643-4516-b712-a0f94ccfbd86-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058037 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058030 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058037 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058038 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058037 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058048 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld5wg\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-kube-api-access-ld5wg\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058335 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058058 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058335 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058068 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058335 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058076 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.058335 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.058085 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c17a6f-5643-4516-b712-a0f94ccfbd86-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:10:48.279627 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.279526 2569 generic.go:356] "Generic (PLEG): container finished" podID="93c17a6f-5643-4516-b712-a0f94ccfbd86" containerID="e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6" exitCode=0 Jul 08 20:10:48.279627 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.279569 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" Jul 08 20:10:48.279627 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.279601 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" event={"ID":"93c17a6f-5643-4516-b712-a0f94ccfbd86","Type":"ContainerDied","Data":"e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6"} Jul 08 20:10:48.279894 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.279663 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9" event={"ID":"93c17a6f-5643-4516-b712-a0f94ccfbd86","Type":"ContainerDied","Data":"6a173fa33ad0e7018d92ea43f727aa56825b28438fdbd883262b56543b668a91"} Jul 08 20:10:48.279894 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.279688 2569 scope.go:117] "RemoveContainer" containerID="e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6" Jul 08 20:10:48.288870 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.288854 2569 scope.go:117] "RemoveContainer" containerID="e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6" Jul 08 20:10:48.289122 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:10:48.289105 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6\": container with ID starting with e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6 not found: ID does not exist" containerID="e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6" Jul 08 20:10:48.289166 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.289131 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6"} err="failed to get container status \"e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6\": rpc error: code = NotFound desc = could not find container \"e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6\": container with ID starting with e258fb7e636417a7debd5dc75a4e65d29e806a20cd3f18f19a45058fe686dec6 not found: ID does not exist" Jul 08 20:10:48.302189 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.302157 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9"] Jul 08 20:10:48.308734 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.308708 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5dd1d121-openshift-default-85ffd9c65f-79hl9"] Jul 08 20:10:48.858476 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.858437 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93c17a6f-5643-4516-b712-a0f94ccfbd86" path="/var/lib/kubelet/pods/93c17a6f-5643-4516-b712-a0f94ccfbd86/volumes" Jul 08 20:10:48.914612 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.914579 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9"] Jul 08 20:10:48.914954 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.914940 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93c17a6f-5643-4516-b712-a0f94ccfbd86" containerName="istio-proxy" Jul 08 20:10:48.914998 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.914956 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="93c17a6f-5643-4516-b712-a0f94ccfbd86" containerName="istio-proxy" Jul 08 20:10:48.915036 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.915021 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="93c17a6f-5643-4516-b712-a0f94ccfbd86" containerName="istio-proxy" Jul 08 20:10:48.919331 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.919305 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.923435 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.923401 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fedc6cb5-openshift-default-dockercfg-5mxck\"" Jul 08 20:10:48.932566 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.932544 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9"] Jul 08 20:10:48.965819 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.965778 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-socket\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.965819 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.965822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb4d7\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-kube-api-access-bb4d7\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966026 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.965912 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-data\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966026 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.965945 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8bdf7e-8905-4600-a273-680edd902a79-istio-podinfo\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966026 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.965982 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-credential-socket\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966026 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.966002 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-istio-token\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966154 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.966053 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-envoy\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966154 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.966076 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-certs\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:48.966154 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:48.966111 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8bdf7e-8905-4600-a273-680edd902a79-istiod-ca-cert\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.066998 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.066961 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-socket\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067221 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067012 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb4d7\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-kube-api-access-bb4d7\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067221 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067044 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-data\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067221 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067173 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8bdf7e-8905-4600-a273-680edd902a79-istio-podinfo\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067400 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-credential-socket\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067400 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067265 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-istio-token\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067400 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067335 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-envoy\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067400 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067377 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-certs\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067599 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8bdf7e-8905-4600-a273-680edd902a79-istiod-ca-cert\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067599 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067479 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-data\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067599 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067525 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-socket\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067739 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067620 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-credential-socket\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.067934 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.067911 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-certs\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.068116 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.068096 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8bdf7e-8905-4600-a273-680edd902a79-istiod-ca-cert\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.069758 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.069733 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8bdf7e-8905-4600-a273-680edd902a79-istio-podinfo\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.069886 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.069829 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-envoy\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.076787 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.076762 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb4d7\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-kube-api-access-bb4d7\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.076979 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.076962 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-istio-token\") pod \"e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.231973 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.231865 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:49.377517 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.377492 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9"] Jul 08 20:10:49.379398 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:10:49.379359 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8bdf7e_8905_4600_a273_680edd902a79.slice/crio-af57fad11b9ed78253c9e9720a944a2b449dc12c34652c3236a9add887615a0e WatchSource:0}: Error finding container af57fad11b9ed78253c9e9720a944a2b449dc12c34652c3236a9add887615a0e: Status 404 returned error can't find the container with id af57fad11b9ed78253c9e9720a944a2b449dc12c34652c3236a9add887615a0e Jul 08 20:10:49.381708 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.381671 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:10:49.381781 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.381758 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:10:49.381819 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:49.381803 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:10:50.293386 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:50.293348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" event={"ID":"2f8bdf7e-8905-4600-a273-680edd902a79","Type":"ContainerStarted","Data":"aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826"} Jul 08 20:10:50.293760 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:50.293403 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" event={"ID":"2f8bdf7e-8905-4600-a273-680edd902a79","Type":"ContainerStarted","Data":"af57fad11b9ed78253c9e9720a944a2b449dc12c34652c3236a9add887615a0e"} Jul 08 20:10:50.330901 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:50.330850 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" podStartSLOduration=2.330832956 podStartE2EDuration="2.330832956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:10:50.328637903 +0000 UTC m=+1745.985745382" watchObservedRunningTime="2026-07-08 20:10:50.330832956 +0000 UTC m=+1745.987940435" Jul 08 20:10:51.232497 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:51.232455 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:51.242273 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:51.242224 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:51.298749 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:51.298710 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:10:51.299744 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:10:51.299717 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:11:01.260931 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:01.260893 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9"] Jul 08 20:11:01.261388 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:01.261105 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" podUID="2f8bdf7e-8905-4600-a273-680edd902a79" containerName="istio-proxy" containerID="cri-o://aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826" gracePeriod=30 Jul 08 20:11:03.839879 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:03.839850 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:11:04.009913 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.009815 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-credential-socket\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.009913 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.009858 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-envoy\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.009913 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.009885 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-data\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.009913 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.009912 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-socket\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.010291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.009928 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8bdf7e-8905-4600-a273-680edd902a79-istiod-ca-cert\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.010291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.009954 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb4d7\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-kube-api-access-bb4d7\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.010291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010011 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-certs\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.010291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010046 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-istio-token\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.010291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010109 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8bdf7e-8905-4600-a273-680edd902a79-istio-podinfo\") pod \"2f8bdf7e-8905-4600-a273-680edd902a79\" (UID: \"2f8bdf7e-8905-4600-a273-680edd902a79\") " Jul 08 20:11:04.010537 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010284 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:04.010537 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010301 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:04.010639 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010547 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.010639 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010554 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f8bdf7e-8905-4600-a273-680edd902a79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:11:04.010639 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010572 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.010862 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010838 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:04.010862 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.010852 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:04.012318 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.012290 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:04.012505 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.012482 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-kube-api-access-bb4d7" (OuterVolumeSpecName: "kube-api-access-bb4d7") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "kube-api-access-bb4d7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:11:04.012610 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.012510 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:11:04.012610 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.012580 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f8bdf7e-8905-4600-a273-680edd902a79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f8bdf7e-8905-4600-a273-680edd902a79" (UID: "2f8bdf7e-8905-4600-a273-680edd902a79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:11:04.111191 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111157 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.111191 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111185 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.111191 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111195 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f8bdf7e-8905-4600-a273-680edd902a79-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.111455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111225 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb4d7\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-kube-api-access-bb4d7\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.111455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111238 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f8bdf7e-8905-4600-a273-680edd902a79-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.111455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111246 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f8bdf7e-8905-4600-a273-680edd902a79-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.111455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.111255 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f8bdf7e-8905-4600-a273-680edd902a79-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:04.366463 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.366422 2569 generic.go:356] "Generic (PLEG): container finished" podID="2f8bdf7e-8905-4600-a273-680edd902a79" containerID="aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826" exitCode=0 Jul 08 20:11:04.366641 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.366470 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" event={"ID":"2f8bdf7e-8905-4600-a273-680edd902a79","Type":"ContainerDied","Data":"aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826"} Jul 08 20:11:04.366641 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.366494 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" Jul 08 20:11:04.366641 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.366554 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9" event={"ID":"2f8bdf7e-8905-4600-a273-680edd902a79","Type":"ContainerDied","Data":"af57fad11b9ed78253c9e9720a944a2b449dc12c34652c3236a9add887615a0e"} Jul 08 20:11:04.366641 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.366586 2569 scope.go:117] "RemoveContainer" containerID="aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826" Jul 08 20:11:04.375876 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.375854 2569 scope.go:117] "RemoveContainer" containerID="aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826" Jul 08 20:11:04.376152 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:11:04.376134 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826\": container with ID starting with aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826 not found: ID does not exist" containerID="aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826" Jul 08 20:11:04.376255 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.376159 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826"} err="failed to get container status \"aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826\": rpc error: code = NotFound desc = could not find container \"aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826\": container with ID starting with aa2fe2dee48b728f845d419efb081f220c2cec4b5037f42486784cd88a226826 not found: ID does not exist" Jul 08 20:11:04.389224 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.389182 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9"] Jul 08 20:11:04.393834 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.393806 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fedc6cb5-openshift-default-5b45b59fcf-xkjt9"] Jul 08 20:11:04.858239 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:04.858182 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f8bdf7e-8905-4600-a273-680edd902a79" path="/var/lib/kubelet/pods/2f8bdf7e-8905-4600-a273-680edd902a79/volumes" Jul 08 20:11:07.515544 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.515518 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9"] Jul 08 20:11:07.515893 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.515858 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f8bdf7e-8905-4600-a273-680edd902a79" containerName="istio-proxy" Jul 08 20:11:07.515893 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.515870 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f8bdf7e-8905-4600-a273-680edd902a79" containerName="istio-proxy" Jul 08 20:11:07.515974 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.515932 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f8bdf7e-8905-4600-a273-680edd902a79" containerName="istio-proxy" Jul 08 20:11:07.518830 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.518813 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.525888 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.525809 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e1c198eb-openshift-default-dockercfg-9hhsp\"" Jul 08 20:11:07.546235 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.546188 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9"] Jul 08 20:11:07.640591 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.640540 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264df592-8de8-4508-a5e0-65bee33258a2-istiod-ca-cert\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.640888 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.640859 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-envoy\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641036 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.640908 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-credential-socket\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641036 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.640937 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsmlq\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-kube-api-access-nsmlq\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.641028 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-certs\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.641066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-data\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.641088 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-istio-token\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.641145 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-socket\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.641418 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.641177 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264df592-8de8-4508-a5e0-65bee33258a2-istio-podinfo\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742341 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742306 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-certs\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742341 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742346 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-data\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742605 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742370 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-istio-token\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742605 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742425 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-socket\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742605 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742456 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264df592-8de8-4508-a5e0-65bee33258a2-istio-podinfo\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742605 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742539 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264df592-8de8-4508-a5e0-65bee33258a2-istiod-ca-cert\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742812 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742780 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-certs\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742868 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742834 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-data\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742868 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742849 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-envoy\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742868 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742859 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-socket\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742997 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742926 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-credential-socket\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.742997 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.742956 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsmlq\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-kube-api-access-nsmlq\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.743269 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.743246 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264df592-8de8-4508-a5e0-65bee33258a2-istiod-ca-cert\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.743368 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.743308 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-credential-socket\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.745012 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.744989 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-envoy\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.745129 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.745108 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264df592-8de8-4508-a5e0-65bee33258a2-istio-podinfo\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.754754 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.754732 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-istio-token\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.755102 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.755071 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsmlq\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-kube-api-access-nsmlq\") pod \"e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.830110 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.830079 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:07.972581 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.972556 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9"] Jul 08 20:11:07.974484 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:11:07.974453 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264df592_8de8_4508_a5e0_65bee33258a2.slice/crio-9af508303c085f642fd60a2a2c111aacee1d1de2172c3d2deee3918c3aeb2d9c WatchSource:0}: Error finding container 9af508303c085f642fd60a2a2c111aacee1d1de2172c3d2deee3918c3aeb2d9c: Status 404 returned error can't find the container with id 9af508303c085f642fd60a2a2c111aacee1d1de2172c3d2deee3918c3aeb2d9c Jul 08 20:11:07.976727 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.976697 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:11:07.976793 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.976764 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:11:07.976834 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:07.976794 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:11:08.387393 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:08.387349 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" event={"ID":"264df592-8de8-4508-a5e0-65bee33258a2","Type":"ContainerStarted","Data":"27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc"} Jul 08 20:11:08.387556 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:08.387446 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" event={"ID":"264df592-8de8-4508-a5e0-65bee33258a2","Type":"ContainerStarted","Data":"9af508303c085f642fd60a2a2c111aacee1d1de2172c3d2deee3918c3aeb2d9c"} Jul 08 20:11:08.408655 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:08.408605 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" podStartSLOduration=1.408588432 podStartE2EDuration="1.408588432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:11:08.408493921 +0000 UTC m=+1764.065601403" watchObservedRunningTime="2026-07-08 20:11:08.408588432 +0000 UTC m=+1764.065695912" Jul 08 20:11:08.830953 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:08.830916 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:09.837952 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:09.837918 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:10.396076 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:10.396040 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:10.397038 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:10.397021 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:13.313658 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:13.313617 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9"] Jul 08 20:11:13.409542 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:13.409497 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" podUID="264df592-8de8-4508-a5e0-65bee33258a2" containerName="istio-proxy" containerID="cri-o://27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc" gracePeriod=30 Jul 08 20:11:18.667777 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.667712 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:18.743338 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743308 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-data\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743338 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743346 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-envoy\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743584 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743379 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-credential-socket\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743584 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743398 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-socket\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743584 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743423 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsmlq\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-kube-api-access-nsmlq\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743584 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743447 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264df592-8de8-4508-a5e0-65bee33258a2-istio-podinfo\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743584 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743473 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-istio-token\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743842 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743589 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264df592-8de8-4508-a5e0-65bee33258a2-istiod-ca-cert\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.743842 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743696 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:18.743842 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743775 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:18.744033 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.743921 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:18.744033 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.744014 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.744124 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.744033 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.744124 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.744052 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.744124 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.744059 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/264df592-8de8-4508-a5e0-65bee33258a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:11:18.745893 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.745863 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:18.746031 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.745941 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-kube-api-access-nsmlq" (OuterVolumeSpecName: "kube-api-access-nsmlq") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "kube-api-access-nsmlq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:11:18.746153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.746136 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:11:18.746153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.746141 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/264df592-8de8-4508-a5e0-65bee33258a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:11:18.844378 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844335 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-certs\") pod \"264df592-8de8-4508-a5e0-65bee33258a2\" (UID: \"264df592-8de8-4508-a5e0-65bee33258a2\") " Jul 08 20:11:18.844547 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844532 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.844547 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844543 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsmlq\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-kube-api-access-nsmlq\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.844624 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844552 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264df592-8de8-4508-a5e0-65bee33258a2-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.844624 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844561 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264df592-8de8-4508-a5e0-65bee33258a2-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.844624 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844569 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264df592-8de8-4508-a5e0-65bee33258a2-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:18.844624 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.844594 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "264df592-8de8-4508-a5e0-65bee33258a2" (UID: "264df592-8de8-4508-a5e0-65bee33258a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:11:18.945519 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:18.945419 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264df592-8de8-4508-a5e0-65bee33258a2-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:11:19.438908 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.438876 2569 generic.go:356] "Generic (PLEG): container finished" podID="264df592-8de8-4508-a5e0-65bee33258a2" containerID="27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc" exitCode=0 Jul 08 20:11:19.439118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.438925 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" Jul 08 20:11:19.439118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.438932 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" event={"ID":"264df592-8de8-4508-a5e0-65bee33258a2","Type":"ContainerDied","Data":"27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc"} Jul 08 20:11:19.439118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.438977 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9" event={"ID":"264df592-8de8-4508-a5e0-65bee33258a2","Type":"ContainerDied","Data":"9af508303c085f642fd60a2a2c111aacee1d1de2172c3d2deee3918c3aeb2d9c"} Jul 08 20:11:19.439118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.438998 2569 scope.go:117] "RemoveContainer" containerID="27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc" Jul 08 20:11:19.449191 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.449171 2569 scope.go:117] "RemoveContainer" containerID="27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc" Jul 08 20:11:19.449517 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:11:19.449489 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc\": container with ID starting with 27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc not found: ID does not exist" containerID="27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc" Jul 08 20:11:19.449610 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.449529 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc"} err="failed to get container status \"27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc\": rpc error: code = NotFound desc = could not find container \"27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc\": container with ID starting with 27c27caed66551e48cda63ef004d72b8af2191eb60ad7a1089754a2d3b4e79bc not found: ID does not exist" Jul 08 20:11:19.458423 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.458399 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9"] Jul 08 20:11:19.465099 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:19.465064 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e1c198eb-openshift-default-7c864956d7-k4tq9"] Jul 08 20:11:20.858751 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:11:20.858715 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="264df592-8de8-4508-a5e0-65bee33258a2" path="/var/lib/kubelet/pods/264df592-8de8-4508-a5e0-65bee33258a2/volumes" Jul 08 20:12:55.595865 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.595812 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz"] Jul 08 20:12:55.596652 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.596220 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="264df592-8de8-4508-a5e0-65bee33258a2" containerName="istio-proxy" Jul 08 20:12:55.596652 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.596234 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="264df592-8de8-4508-a5e0-65bee33258a2" containerName="istio-proxy" Jul 08 20:12:55.596652 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.596298 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="264df592-8de8-4508-a5e0-65bee33258a2" containerName="istio-proxy" Jul 08 20:12:55.598320 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.598303 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.601419 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.601395 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-89ad9d7c-openshift-default-dockercfg-bdnz9\"" Jul 08 20:12:55.623746 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.623718 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz"] Jul 08 20:12:55.676888 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.676852 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhd6h\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-kube-api-access-xhd6h\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.676888 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.676892 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-certs\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.676914 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e6643-5143-4afc-a794-e5ffff9b3352-istiod-ca-cert\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.676998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-podinfo\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.677047 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-credential-socket\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.677070 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-data\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.677101 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-token\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677367 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.677130 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-socket\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.677367 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.677151 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-envoy\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778329 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778295 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-certs\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778329 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778337 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e6643-5143-4afc-a794-e5ffff9b3352-istiod-ca-cert\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778360 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-podinfo\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-credential-socket\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-data\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-token\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-socket\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778479 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-envoy\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778557 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhd6h\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-kube-api-access-xhd6h\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778973 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778847 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-credential-socket\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778973 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-certs\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.778973 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778921 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-socket\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.779458 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.778986 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-data\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.779458 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.779174 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e6643-5143-4afc-a794-e5ffff9b3352-istiod-ca-cert\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.780854 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.780830 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-envoy\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.781062 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.781043 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-podinfo\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.787807 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.787784 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-token\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.788089 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.788067 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhd6h\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-kube-api-access-xhd6h\") pod \"e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:55.908053 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:55.907955 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:56.063840 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.063802 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz"] Jul 08 20:12:56.066551 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:12:56.066514 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18e6643_5143_4afc_a794_e5ffff9b3352.slice/crio-9afbddc8f3d40aa02d01697c246a8bedfa3f843c0128cf8ea273c4e6e8e17c59 WatchSource:0}: Error finding container 9afbddc8f3d40aa02d01697c246a8bedfa3f843c0128cf8ea273c4e6e8e17c59: Status 404 returned error can't find the container with id 9afbddc8f3d40aa02d01697c246a8bedfa3f843c0128cf8ea273c4e6e8e17c59 Jul 08 20:12:56.068836 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.068798 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:12:56.068916 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.068895 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:12:56.068962 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.068934 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:12:56.893845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.893807 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" event={"ID":"b18e6643-5143-4afc-a794-e5ffff9b3352","Type":"ContainerStarted","Data":"2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047"} Jul 08 20:12:56.894284 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.893862 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" event={"ID":"b18e6643-5143-4afc-a794-e5ffff9b3352","Type":"ContainerStarted","Data":"9afbddc8f3d40aa02d01697c246a8bedfa3f843c0128cf8ea273c4e6e8e17c59"} Jul 08 20:12:56.908296 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.908271 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:56.924582 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:56.920401 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" podStartSLOduration=1.920380778 podStartE2EDuration="1.920380778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:12:56.917726992 +0000 UTC m=+1872.574834486" watchObservedRunningTime="2026-07-08 20:12:56.920380778 +0000 UTC m=+1872.577488259" Jul 08 20:12:57.914135 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:57.914105 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:58.903499 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:58.903466 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:12:58.904539 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:12:58.904522 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:13:12.975827 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:12.975737 2569 generic.go:356] "Generic (PLEG): container finished" podID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerID="2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047" exitCode=0 Jul 08 20:13:12.975827 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:12.975805 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" event={"ID":"b18e6643-5143-4afc-a794-e5ffff9b3352","Type":"ContainerDied","Data":"2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047"} Jul 08 20:13:12.976309 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:12.976177 2569 scope.go:117] "RemoveContainer" containerID="2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047" Jul 08 20:13:12.977172 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:12.977144 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:12.977320 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:12.977245 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:12.977320 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:12.977297 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:13.903805 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:13.903769 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:13:13.981973 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:13.981939 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" event={"ID":"b18e6643-5143-4afc-a794-e5ffff9b3352","Type":"ContainerStarted","Data":"602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13"} Jul 08 20:13:14.908774 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:14.908729 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:13:14.910034 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:14.910002 2569 patch_prober.go:28] interesting pod/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 20:13:14.910171 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:14.910056 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 20:13:15.909091 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:15.909051 2569 patch_prober.go:28] interesting pod/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 20:13:15.909536 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:15.909110 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 20:13:16.908914 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:16.908874 2569 patch_prober.go:28] interesting pod/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 08 20:13:16.909114 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:16.908944 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 08 20:13:17.065556 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:17.065516 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz"] Jul 08 20:13:17.066135 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:17.066085 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" containerID="cri-o://602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13" gracePeriod=30 Jul 08 20:13:23.021651 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.021626 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:13:23.024016 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.023986 2569 generic.go:356] "Generic (PLEG): container finished" podID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerID="602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13" exitCode=0 Jul 08 20:13:23.024137 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.024036 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" Jul 08 20:13:23.024137 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.024061 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" event={"ID":"b18e6643-5143-4afc-a794-e5ffff9b3352","Type":"ContainerDied","Data":"602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13"} Jul 08 20:13:23.024137 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.024109 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz" event={"ID":"b18e6643-5143-4afc-a794-e5ffff9b3352","Type":"ContainerDied","Data":"9afbddc8f3d40aa02d01697c246a8bedfa3f843c0128cf8ea273c4e6e8e17c59"} Jul 08 20:13:23.024137 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.024132 2569 scope.go:117] "RemoveContainer" containerID="602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13" Jul 08 20:13:23.032472 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.032450 2569 scope.go:117] "RemoveContainer" containerID="2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047" Jul 08 20:13:23.040365 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.040344 2569 scope.go:117] "RemoveContainer" containerID="602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13" Jul 08 20:13:23.040662 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:13:23.040640 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13\": container with ID starting with 602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13 not found: ID does not exist" containerID="602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13" Jul 08 20:13:23.040716 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.040674 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13"} err="failed to get container status \"602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13\": rpc error: code = NotFound desc = could not find container \"602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13\": container with ID starting with 602f8b2752d8cd8b36a60168df3090f59577e479d34ac09b485ce98546a99b13 not found: ID does not exist" Jul 08 20:13:23.040716 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.040700 2569 scope.go:117] "RemoveContainer" containerID="2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047" Jul 08 20:13:23.040976 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:13:23.040957 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047\": container with ID starting with 2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047 not found: ID does not exist" containerID="2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047" Jul 08 20:13:23.041020 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.040984 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047"} err="failed to get container status \"2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047\": rpc error: code = NotFound desc = could not find container \"2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047\": container with ID starting with 2ef33f6905233c874f702b2e7b4797017752d3949616127c3db5c0f890097047 not found: ID does not exist" Jul 08 20:13:23.130989 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.130891 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-data\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.130989 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.130939 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-socket\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.130989 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.130970 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhd6h\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-kube-api-access-xhd6h\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131035 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-podinfo\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131066 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-certs\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131086 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-credential-socket\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131155 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e6643-5143-4afc-a794-e5ffff9b3352-istiod-ca-cert\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131178 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-token\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131240 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-envoy\") pod \"b18e6643-5143-4afc-a794-e5ffff9b3352\" (UID: \"b18e6643-5143-4afc-a794-e5ffff9b3352\") " Jul 08 20:13:23.131304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131288 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:13:23.131654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131444 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:13:23.131654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131465 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:13:23.131654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131555 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.131654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131562 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18e6643-5143-4afc-a794-e5ffff9b3352-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:13:23.131654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131573 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.131654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.131605 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.132390 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.132360 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-data" (OuterVolumeSpecName: "istio-data") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:13:23.133495 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.133469 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-token" (OuterVolumeSpecName: "istio-token") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:13:23.133622 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.133493 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-kube-api-access-xhd6h" (OuterVolumeSpecName: "kube-api-access-xhd6h") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "kube-api-access-xhd6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:13:23.133888 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.133868 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:13:23.133965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.133867 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b18e6643-5143-4afc-a794-e5ffff9b3352" (UID: "b18e6643-5143-4afc-a794-e5ffff9b3352"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:13:23.232904 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.232865 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e6643-5143-4afc-a794-e5ffff9b3352-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.232904 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.232896 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.232904 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.232905 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.232904 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.232913 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.233165 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.232943 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhd6h\" (UniqueName: \"kubernetes.io/projected/b18e6643-5143-4afc-a794-e5ffff9b3352-kube-api-access-xhd6h\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.233165 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.232954 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e6643-5143-4afc-a794-e5ffff9b3352-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:13:23.348970 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.348911 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz"] Jul 08 20:13:23.350795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:23.350770 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-89ad9d7c-openshift-default-58976cc755-jrxsz"] Jul 08 20:13:24.858642 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:24.858602 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" path="/var/lib/kubelet/pods/b18e6643-5143-4afc-a794-e5ffff9b3352/volumes" Jul 08 20:13:36.651763 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.651724 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh"] Jul 08 20:13:36.652397 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.652118 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" Jul 08 20:13:36.652397 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.652131 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" Jul 08 20:13:36.652397 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.652223 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" Jul 08 20:13:36.655013 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.654993 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.658589 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.658557 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-28e49818-openshift-default-dockercfg-vgh9f\"" Jul 08 20:13:36.673708 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.673682 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh"] Jul 08 20:13:36.751416 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751381 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mblk\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-kube-api-access-8mblk\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751610 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751436 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-certs\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751610 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751518 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-envoy\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751734 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751611 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-data\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751734 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751655 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-credential-socket\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751734 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751696 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-istio-token\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751879 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751730 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67779509-e504-449c-b1b5-8cd60fe381b2-istio-podinfo\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751879 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751845 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-socket\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.751879 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.751877 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67779509-e504-449c-b1b5-8cd60fe381b2-istiod-ca-cert\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.852974 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.852939 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mblk\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-kube-api-access-8mblk\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.852985 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-certs\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853031 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-envoy\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853057 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-data\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-credential-socket\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853134 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-istio-token\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853167 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67779509-e504-449c-b1b5-8cd60fe381b2-istio-podinfo\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853552 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-socket\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853552 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67779509-e504-449c-b1b5-8cd60fe381b2-istiod-ca-cert\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853552 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853464 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-certs\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853747 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853546 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-data\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853747 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853650 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-credential-socket\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.853747 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.853711 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-socket\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.854050 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.854029 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67779509-e504-449c-b1b5-8cd60fe381b2-istiod-ca-cert\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.855471 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.855446 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-envoy\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.855753 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.855728 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67779509-e504-449c-b1b5-8cd60fe381b2-istio-podinfo\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.862901 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.862871 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-istio-token\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.863033 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.862949 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mblk\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-kube-api-access-8mblk\") pod \"e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:36.965159 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:36.965069 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:37.110552 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:37.110524 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh"] Jul 08 20:13:37.112585 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:13:37.112553 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice/crio-1845eade9654754b676bbab3ff3a1dd33c101c4eb7a80129c13c34a9f3031d86 WatchSource:0}: Error finding container 1845eade9654754b676bbab3ff3a1dd33c101c4eb7a80129c13c34a9f3031d86: Status 404 returned error can't find the container with id 1845eade9654754b676bbab3ff3a1dd33c101c4eb7a80129c13c34a9f3031d86 Jul 08 20:13:37.114586 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:37.114556 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:37.114654 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:37.114629 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:37.114689 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:37.114659 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:38.098339 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:38.098302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" event={"ID":"67779509-e504-449c-b1b5-8cd60fe381b2","Type":"ContainerStarted","Data":"746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0"} Jul 08 20:13:38.098787 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:38.098361 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" event={"ID":"67779509-e504-449c-b1b5-8cd60fe381b2","Type":"ContainerStarted","Data":"1845eade9654754b676bbab3ff3a1dd33c101c4eb7a80129c13c34a9f3031d86"} Jul 08 20:13:38.120348 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:38.120297 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podStartSLOduration=2.120278831 podStartE2EDuration="2.120278831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:13:38.117937659 +0000 UTC m=+1913.775045154" watchObservedRunningTime="2026-07-08 20:13:38.120278831 +0000 UTC m=+1913.777386311" Jul 08 20:13:38.965590 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:38.965531 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:38.970528 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:38.970499 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:39.104607 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:39.104576 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:39.105536 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:39.105518 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:54.471266 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:54.470823 2569 patch_prober.go:28] interesting pod/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:44586->10.132.0.44:15021: read: connection reset by peer" start-of-body= Jul 08 20:13:54.471266 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:54.470905 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:44586->10.132.0.44:15021: read: connection reset by peer" Jul 08 20:13:55.192771 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:55.192735 2569 generic.go:356] "Generic (PLEG): container finished" podID="67779509-e504-449c-b1b5-8cd60fe381b2" containerID="746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0" exitCode=0 Jul 08 20:13:55.192985 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:55.192810 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" event={"ID":"67779509-e504-449c-b1b5-8cd60fe381b2","Type":"ContainerDied","Data":"746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0"} Jul 08 20:13:55.193224 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:55.193194 2569 scope.go:117] "RemoveContainer" containerID="746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0" Jul 08 20:13:55.194188 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:55.194159 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:55.194314 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:55.194252 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:55.194314 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:55.194289 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:13:56.199141 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:56.199104 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" event={"ID":"67779509-e504-449c-b1b5-8cd60fe381b2","Type":"ContainerStarted","Data":"4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29"} Jul 08 20:13:56.965303 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:56.965261 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:13:56.965569 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:56.965545 2569 patch_prober.go:28] interesting pod/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 20:13:56.965618 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:56.965597 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 20:13:57.966120 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:57.966075 2569 patch_prober.go:28] interesting pod/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 20:13:57.966611 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:57.966152 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 20:13:58.966221 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:58.966164 2569 patch_prober.go:28] interesting pod/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 20:13:58.966644 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:58.966246 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 20:13:59.965676 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:59.965632 2569 patch_prober.go:28] interesting pod/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 08 20:13:59.965843 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:13:59.965696 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 08 20:14:00.969884 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:14:00.969854 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:14:00.970353 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:14:00.970112 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:14:00.970801 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:14:00.970785 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:15:39.305418 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:39.305379 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh"] Jul 08 20:15:39.305992 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:39.305620 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" containerID="cri-o://4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29" gracePeriod=30 Jul 08 20:15:44.565618 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.565585 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:15:44.568929 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.568872 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-credential-socket\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569028 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.568944 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67779509-e504-449c-b1b5-8cd60fe381b2-istio-podinfo\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569076 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569032 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-envoy\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569148 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569129 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-certs\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569194 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569168 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mblk\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-kube-api-access-8mblk\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569278 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569192 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-socket\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569278 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569246 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-istio-token\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569386 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569266 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:15:44.569386 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569292 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-data\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569386 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569318 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67779509-e504-449c-b1b5-8cd60fe381b2-istiod-ca-cert\") pod \"67779509-e504-449c-b1b5-8cd60fe381b2\" (UID: \"67779509-e504-449c-b1b5-8cd60fe381b2\") " Jul 08 20:15:44.569541 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569501 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:15:44.569541 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569512 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:15:44.569693 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569671 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.569767 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569697 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.569767 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569712 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.569876 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.569809 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67779509-e504-449c-b1b5-8cd60fe381b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:15:44.570348 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.570309 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:15:44.578569 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.571849 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67779509-e504-449c-b1b5-8cd60fe381b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:15:44.580922 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.580876 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:15:44.580922 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.580869 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-kube-api-access-8mblk" (OuterVolumeSpecName: "kube-api-access-8mblk") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "kube-api-access-8mblk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:15:44.581106 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.581043 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67779509-e504-449c-b1b5-8cd60fe381b2" (UID: "67779509-e504-449c-b1b5-8cd60fe381b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:15:44.670553 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.670512 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67779509-e504-449c-b1b5-8cd60fe381b2-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.670553 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.670545 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.670553 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.670559 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mblk\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-kube-api-access-8mblk\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.670800 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.670572 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67779509-e504-449c-b1b5-8cd60fe381b2-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.670800 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.670583 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67779509-e504-449c-b1b5-8cd60fe381b2-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.670800 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.670594 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67779509-e504-449c-b1b5-8cd60fe381b2-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:15:44.722160 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.722124 2569 generic.go:356] "Generic (PLEG): container finished" podID="67779509-e504-449c-b1b5-8cd60fe381b2" containerID="4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29" exitCode=0 Jul 08 20:15:44.722362 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.722181 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" Jul 08 20:15:44.722362 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.722195 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" event={"ID":"67779509-e504-449c-b1b5-8cd60fe381b2","Type":"ContainerDied","Data":"4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29"} Jul 08 20:15:44.722362 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.722352 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh" event={"ID":"67779509-e504-449c-b1b5-8cd60fe381b2","Type":"ContainerDied","Data":"1845eade9654754b676bbab3ff3a1dd33c101c4eb7a80129c13c34a9f3031d86"} Jul 08 20:15:44.722490 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.722377 2569 scope.go:117] "RemoveContainer" containerID="4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29" Jul 08 20:15:44.732340 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.732319 2569 scope.go:117] "RemoveContainer" containerID="746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0" Jul 08 20:15:44.741075 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.741057 2569 scope.go:117] "RemoveContainer" containerID="4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29" Jul 08 20:15:44.741359 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:15:44.741338 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29\": container with ID starting with 4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29 not found: ID does not exist" containerID="4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29" Jul 08 20:15:44.741417 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.741368 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29"} err="failed to get container status \"4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29\": rpc error: code = NotFound desc = could not find container \"4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29\": container with ID starting with 4578fda3ec43262721a231e1db5412982db5afe0b55995a79bf85ded4b944e29 not found: ID does not exist" Jul 08 20:15:44.741417 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.741386 2569 scope.go:117] "RemoveContainer" containerID="746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0" Jul 08 20:15:44.741668 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:15:44.741649 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0\": container with ID starting with 746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0 not found: ID does not exist" containerID="746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0" Jul 08 20:15:44.741712 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.741675 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0"} err="failed to get container status \"746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0\": rpc error: code = NotFound desc = could not find container \"746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0\": container with ID starting with 746ec107fd9e6305b406ec2610e4ca1b00516c929864c5e1c513fc741e374fa0 not found: ID does not exist" Jul 08 20:15:44.750587 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.750557 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh"] Jul 08 20:15:44.755216 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.755163 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-28e49818-openshift-default-6579cbc65f-l6lvh"] Jul 08 20:15:44.860688 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:15:44.860589 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" path="/var/lib/kubelet/pods/67779509-e504-449c-b1b5-8cd60fe381b2/volumes" Jul 08 20:15:50.679671 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:15:50.679635 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:15:51.330559 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:15:51.330520 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:00.721221 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:00.721171 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:06.363446 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:06.363406 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:10.730118 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:10.730034 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:20.771557 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:20.771521 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:21.315821 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:21.315772 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:24.967907 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:24.967859 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:24.968556 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:24.968527 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:30.812842 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:30.812805 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:33.653984 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.653949 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4"] Jul 08 20:16:33.654551 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654530 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" Jul 08 20:16:33.654638 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654555 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" Jul 08 20:16:33.654638 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654574 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" Jul 08 20:16:33.654638 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654583 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" Jul 08 20:16:33.654638 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654595 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" Jul 08 20:16:33.654638 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654603 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" Jul 08 20:16:33.654890 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654691 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" Jul 08 20:16:33.654890 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654704 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18e6643-5143-4afc-a794-e5ffff9b3352" containerName="istio-proxy" Jul 08 20:16:33.654890 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.654718 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="67779509-e504-449c-b1b5-8cd60fe381b2" containerName="istio-proxy" Jul 08 20:16:33.657862 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.657840 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.661255 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.661231 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-db100e-openshift-default-dockercfg-g6v42\"" Jul 08 20:16:33.669747 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.669723 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4"] Jul 08 20:16:33.695759 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695709 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-data\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.695942 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695771 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66347b6-d407-485e-8df2-893f425b74e7-istiod-ca-cert\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.695942 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-socket\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.695942 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695858 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-istio-token\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.695942 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695901 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-credential-socket\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.695942 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695930 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66347b6-d407-485e-8df2-893f425b74e7-istio-podinfo\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.696110 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695950 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-envoy\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.696110 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.695991 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-certs\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.696110 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.696013 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64z9p\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-kube-api-access-64z9p\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796492 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796452 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-certs\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796682 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796497 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64z9p\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-kube-api-access-64z9p\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796682 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796550 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-data\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796682 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66347b6-d407-485e-8df2-893f425b74e7-istiod-ca-cert\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796792 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796728 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-socket\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796792 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796777 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-istio-token\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796883 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-credential-socket\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796883 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796866 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66347b6-d407-485e-8df2-893f425b74e7-istio-podinfo\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796980 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796896 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-envoy\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796980 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796931 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-certs\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.796980 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.796972 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-data\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.797120 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.797051 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-socket\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.797216 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.797179 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-credential-socket\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.797364 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.797336 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66347b6-d407-485e-8df2-893f425b74e7-istiod-ca-cert\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.799127 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.799107 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-envoy\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.799374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.799354 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66347b6-d407-485e-8df2-893f425b74e7-istio-podinfo\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.804655 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.804634 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-istio-token\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.804844 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.804821 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64z9p\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-kube-api-access-64z9p\") pod \"e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:33.968649 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:33.968555 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:34.113941 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.113915 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4"] Jul 08 20:16:34.115975 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:16:34.115948 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66347b6_d407_485e_8df2_893f425b74e7.slice/crio-072db17f83be0c0021657411d6d90929a0021dc707d46d7691b583732c31de2d WatchSource:0}: Error finding container 072db17f83be0c0021657411d6d90929a0021dc707d46d7691b583732c31de2d: Status 404 returned error can't find the container with id 072db17f83be0c0021657411d6d90929a0021dc707d46d7691b583732c31de2d Jul 08 20:16:34.118757 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.118719 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:34.118836 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.118807 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:34.118874 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.118852 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:34.958414 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.958377 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" event={"ID":"a66347b6-d407-485e-8df2-893f425b74e7","Type":"ContainerStarted","Data":"b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b"} Jul 08 20:16:34.958800 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.958432 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" event={"ID":"a66347b6-d407-485e-8df2-893f425b74e7","Type":"ContainerStarted","Data":"072db17f83be0c0021657411d6d90929a0021dc707d46d7691b583732c31de2d"} Jul 08 20:16:34.968770 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.968741 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:34.979706 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:34.979656 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" podStartSLOduration=1.979638298 podStartE2EDuration="1.979638298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:16:34.978087066 +0000 UTC m=+2090.635194541" watchObservedRunningTime="2026-07-08 20:16:34.979638298 +0000 UTC m=+2090.636745779" Jul 08 20:16:35.974535 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:35.974495 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:36.311550 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:36.311516 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:36.968265 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:36.968233 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:36.969134 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:36.969111 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:40.855337 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:16:40.855304 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67779509_e504_449c_b1b5_8cd60fe381b2.slice\": RecentStats: unable to find data in memory cache]" Jul 08 20:16:41.972944 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:41.972909 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph"] Jul 08 20:16:41.976758 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:41.976735 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:41.982605 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:41.982583 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4eb9fe-openshift-default-dockercfg-h5g2x\"" Jul 08 20:16:41.992645 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:41.992618 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph"] Jul 08 20:16:42.071928 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.071895 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-credential-socket\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.071928 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.071933 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-podinfo\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.071972 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-socket\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.071994 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8pfj\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-kube-api-access-t8pfj\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.072064 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-data\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.072096 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-envoy\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.072116 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-token\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.072143 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a8a9eec-a25f-4626-bb15-f1447a56160b-istiod-ca-cert\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.072187 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.072161 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-certs\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173365 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173326 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-credential-socket\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173582 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173370 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-podinfo\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173582 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173429 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-socket\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173582 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173455 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8pfj\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-kube-api-access-t8pfj\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173582 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173514 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-data\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173582 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-envoy\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173870 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173585 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-token\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173870 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173617 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a8a9eec-a25f-4626-bb15-f1447a56160b-istiod-ca-cert\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.173870 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173646 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-certs\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.174001 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173869 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-socket\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.174001 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.173902 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-data\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.174071 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.174038 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-credential-socket\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.174164 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.174146 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-certs\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.174498 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.174473 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a8a9eec-a25f-4626-bb15-f1447a56160b-istiod-ca-cert\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.175929 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.175908 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-envoy\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.176060 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.176043 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-podinfo\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.181424 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.181399 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-token\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.181564 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.181543 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8pfj\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-kube-api-access-t8pfj\") pod \"e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.290105 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.289998 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:42.434917 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.434876 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph"] Jul 08 20:16:42.436761 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:16:42.436722 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8a9eec_a25f_4626_bb15_f1447a56160b.slice/crio-bf74bb55fed87cce83e47b0f84fbddb96b5a85f88787227bde2171aa1603eb51 WatchSource:0}: Error finding container bf74bb55fed87cce83e47b0f84fbddb96b5a85f88787227bde2171aa1603eb51: Status 404 returned error can't find the container with id bf74bb55fed87cce83e47b0f84fbddb96b5a85f88787227bde2171aa1603eb51 Jul 08 20:16:42.439615 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.439577 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:42.439721 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.439666 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:42.439721 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.439714 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:42.997765 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.997728 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" event={"ID":"9a8a9eec-a25f-4626-bb15-f1447a56160b","Type":"ContainerStarted","Data":"2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884"} Jul 08 20:16:42.998312 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:42.997800 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" event={"ID":"9a8a9eec-a25f-4626-bb15-f1447a56160b","Type":"ContainerStarted","Data":"bf74bb55fed87cce83e47b0f84fbddb96b5a85f88787227bde2171aa1603eb51"} Jul 08 20:16:43.022131 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:43.022078 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podStartSLOduration=2.022057591 podStartE2EDuration="2.022057591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:16:43.020917426 +0000 UTC m=+2098.678024918" watchObservedRunningTime="2026-07-08 20:16:43.022057591 +0000 UTC m=+2098.679165071" Jul 08 20:16:43.290584 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:43.290493 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:43.295468 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:43.295443 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:44.002647 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:44.002614 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:44.003804 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:44.003784 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:16:53.009421 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:53.009378 2569 patch_prober.go:28] interesting pod/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 20:16:53.009817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:53.009452 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 20:16:54.052637 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:54.052603 2569 generic.go:356] "Generic (PLEG): container finished" podID="a66347b6-d407-485e-8df2-893f425b74e7" containerID="b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b" exitCode=0 Jul 08 20:16:54.053064 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:54.052680 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" event={"ID":"a66347b6-d407-485e-8df2-893f425b74e7","Type":"ContainerDied","Data":"b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b"} Jul 08 20:16:54.053105 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:54.053090 2569 scope.go:117] "RemoveContainer" containerID="b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b" Jul 08 20:16:54.054116 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:54.054090 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:54.054178 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:54.054165 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:54.054240 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:54.054226 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:16:55.061785 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:55.061735 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" event={"ID":"a66347b6-d407-485e-8df2-893f425b74e7","Type":"ContainerStarted","Data":"794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d"} Jul 08 20:16:55.969851 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:55.969810 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:55.970273 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:55.970247 2569 patch_prober.go:28] interesting pod/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 20:16:55.970464 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:55.970439 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 20:16:56.969522 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:56.969469 2569 patch_prober.go:28] interesting pod/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 20:16:56.970047 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:56.969549 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 20:16:57.969325 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:57.969275 2569 patch_prober.go:28] interesting pod/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 08 20:16:57.969556 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:57.969350 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 08 20:16:58.976382 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:58.976345 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:58.976882 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:58.976617 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:16:58.977295 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:16:58.977275 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:17:03.109699 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:03.109663 2569 generic.go:356] "Generic (PLEG): container finished" podID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerID="2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884" exitCode=0 Jul 08 20:17:03.110121 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:03.109717 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" event={"ID":"9a8a9eec-a25f-4626-bb15-f1447a56160b","Type":"ContainerDied","Data":"2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884"} Jul 08 20:17:03.110121 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:03.110100 2569 scope.go:117] "RemoveContainer" containerID="2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884" Jul 08 20:17:03.111097 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:03.111070 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:17:03.111154 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:03.111141 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:17:03.111189 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:03.111178 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:17:04.116642 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:04.116609 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" event={"ID":"9a8a9eec-a25f-4626-bb15-f1447a56160b","Type":"ContainerStarted","Data":"3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747"} Jul 08 20:17:04.290222 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:04.290165 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:17:04.290437 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:04.290415 2569 patch_prober.go:28] interesting pod/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 20:17:04.290484 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:04.290461 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 20:17:05.290502 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:05.290468 2569 patch_prober.go:28] interesting pod/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 20:17:05.290902 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:05.290518 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 20:17:06.291036 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:06.290996 2569 patch_prober.go:28] interesting pod/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 20:17:06.291483 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:06.291055 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 20:17:07.290986 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:07.290943 2569 patch_prober.go:28] interesting pod/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 20:17:07.291184 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:07.291007 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 20:17:08.291104 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:08.291061 2569 patch_prober.go:28] interesting pod/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 08 20:17:08.291425 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:08.291130 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 08 20:17:09.295563 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:09.295532 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:17:09.295956 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:09.295809 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:17:09.296478 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:17:09.296463 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:18:21.155818 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:21.155776 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4"] Jul 08 20:18:21.156305 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:21.156051 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" containerID="cri-o://794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d" gracePeriod=30 Jul 08 20:18:26.412640 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.412614 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:18:26.510341 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510252 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-envoy\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510341 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510293 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66347b6-d407-485e-8df2-893f425b74e7-istiod-ca-cert\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510341 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510327 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66347b6-d407-485e-8df2-893f425b74e7-istio-podinfo\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510348 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-istio-token\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510397 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-data\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510415 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64z9p\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-kube-api-access-64z9p\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510453 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-credential-socket\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510492 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-socket\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510550 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-certs\") pod \"a66347b6-d407-485e-8df2-893f425b74e7\" (UID: \"a66347b6-d407-485e-8df2-893f425b74e7\") " Jul 08 20:18:26.510928 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510765 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66347b6-d407-485e-8df2-893f425b74e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:18:26.510928 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510849 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:26.511053 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.510986 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:26.511112 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.511049 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:26.511600 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.511572 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:26.512679 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.512653 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:18:26.512805 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.512790 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-kube-api-access-64z9p" (OuterVolumeSpecName: "kube-api-access-64z9p") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "kube-api-access-64z9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:18:26.512948 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.512935 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:26.512990 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.512969 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a66347b6-d407-485e-8df2-893f425b74e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a66347b6-d407-485e-8df2-893f425b74e7" (UID: "a66347b6-d407-485e-8df2-893f425b74e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:18:26.519812 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.519787 2569 generic.go:356] "Generic (PLEG): container finished" podID="a66347b6-d407-485e-8df2-893f425b74e7" containerID="794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d" exitCode=0 Jul 08 20:18:26.519905 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.519851 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" Jul 08 20:18:26.519905 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.519849 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" event={"ID":"a66347b6-d407-485e-8df2-893f425b74e7","Type":"ContainerDied","Data":"794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d"} Jul 08 20:18:26.519993 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.519951 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4" event={"ID":"a66347b6-d407-485e-8df2-893f425b74e7","Type":"ContainerDied","Data":"072db17f83be0c0021657411d6d90929a0021dc707d46d7691b583732c31de2d"} Jul 08 20:18:26.519993 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.519983 2569 scope.go:117] "RemoveContainer" containerID="794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d" Jul 08 20:18:26.531750 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.531732 2569 scope.go:117] "RemoveContainer" containerID="b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b" Jul 08 20:18:26.540899 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.540880 2569 scope.go:117] "RemoveContainer" containerID="794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d" Jul 08 20:18:26.541251 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:18:26.541223 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d\": container with ID starting with 794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d not found: ID does not exist" containerID="794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d" Jul 08 20:18:26.541251 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.541265 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d"} err="failed to get container status \"794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d\": rpc error: code = NotFound desc = could not find container \"794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d\": container with ID starting with 794022a80f3421af24c1aa5318e95679e6af3ed010192bff449858908070a44d not found: ID does not exist" Jul 08 20:18:26.541431 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.541291 2569 scope.go:117] "RemoveContainer" containerID="b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b" Jul 08 20:18:26.541667 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:18:26.541626 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b\": container with ID starting with b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b not found: ID does not exist" containerID="b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b" Jul 08 20:18:26.541778 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.541659 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b"} err="failed to get container status \"b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b\": rpc error: code = NotFound desc = could not find container \"b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b\": container with ID starting with b84f8a4f9ad80524176e6b40aa9e1f95b88422f06b97403966a6a38a2e26640b not found: ID does not exist" Jul 08 20:18:26.543687 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.543668 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4"] Jul 08 20:18:26.548027 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.548006 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-db100e-openshift-default-555b949c49-tsrf4"] Jul 08 20:18:26.612058 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612021 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612058 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612046 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64z9p\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-kube-api-access-64z9p\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612058 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612058 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612069 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612078 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612086 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66347b6-d407-485e-8df2-893f425b74e7-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612093 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66347b6-d407-485e-8df2-893f425b74e7-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612100 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66347b6-d407-485e-8df2-893f425b74e7-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.612374 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.612107 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66347b6-d407-485e-8df2-893f425b74e7-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:26.858642 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:26.858609 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66347b6-d407-485e-8df2-893f425b74e7" path="/var/lib/kubelet/pods/a66347b6-d407-485e-8df2-893f425b74e7/volumes" Jul 08 20:18:27.936063 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:27.936024 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph"] Jul 08 20:18:27.936610 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:27.936307 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" containerID="cri-o://3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747" gracePeriod=30 Jul 08 20:18:28.790434 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.790395 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz"] Jul 08 20:18:28.790775 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.790763 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" Jul 08 20:18:28.790816 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.790778 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" Jul 08 20:18:28.790863 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.790853 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" Jul 08 20:18:28.795118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.795101 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.797785 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.797764 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-85eb30-openshift-default-dockercfg-qtpv8\"" Jul 08 20:18:28.804249 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.804221 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz"] Jul 08 20:18:28.829178 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829146 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istiod-ca-cert\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829362 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-socket\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829362 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829256 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-envoy\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829367 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-credential-socket\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829427 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-token\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829524 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829465 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-certs\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829524 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829494 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-podinfo\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829595 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829524 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-data\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.829595 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.829557 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9lk4\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-kube-api-access-k9lk4\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.930679 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.930636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-token\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.930679 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.930679 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-certs\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.930948 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.930829 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-podinfo\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.930948 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.930886 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-data\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.930948 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.930926 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9lk4\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-kube-api-access-k9lk4\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.931111 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931007 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istiod-ca-cert\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.931111 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931061 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-socket\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.931111 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-envoy\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.931297 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931127 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-credential-socket\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.931297 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931172 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-certs\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.931487 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931334 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-data\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.938236 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.931910 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-socket\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.938236 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.932496 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istiod-ca-cert\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.938236 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.932721 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-credential-socket\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.938236 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.935161 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-envoy\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.938236 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.938136 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-podinfo\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.939602 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.939534 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-token\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:28.940631 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:28.940589 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9lk4\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-kube-api-access-k9lk4\") pod \"e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:29.105629 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.105588 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:29.241135 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.241111 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz"] Jul 08 20:18:29.243534 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:18:29.243501 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f11df6a_5773_4a15_90b9_6e933c2cdebb.slice/crio-fab4ab72e49eed7e3fd4aebc3068b3599e8f1840529a152562af295cc223193a WatchSource:0}: Error finding container fab4ab72e49eed7e3fd4aebc3068b3599e8f1840529a152562af295cc223193a: Status 404 returned error can't find the container with id fab4ab72e49eed7e3fd4aebc3068b3599e8f1840529a152562af295cc223193a Jul 08 20:18:29.245717 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.245681 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:29.245804 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.245765 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:29.245845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.245804 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:29.537394 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.537296 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" event={"ID":"1f11df6a-5773-4a15-90b9-6e933c2cdebb","Type":"ContainerStarted","Data":"39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0"} Jul 08 20:18:29.537394 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.537362 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" event={"ID":"1f11df6a-5773-4a15-90b9-6e933c2cdebb","Type":"ContainerStarted","Data":"fab4ab72e49eed7e3fd4aebc3068b3599e8f1840529a152562af295cc223193a"} Jul 08 20:18:29.558656 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:29.558601 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podStartSLOduration=1.558583096 podStartE2EDuration="1.558583096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:18:29.557388932 +0000 UTC m=+2205.214496425" watchObservedRunningTime="2026-07-08 20:18:29.558583096 +0000 UTC m=+2205.215690575" Jul 08 20:18:30.106190 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:30.106145 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:30.107700 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:30.107675 2569 patch_prober.go:28] interesting pod/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 20:18:30.107764 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:30.107733 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 20:18:31.110467 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:31.110433 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:31.548400 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:31.548316 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:31.549637 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:31.549609 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:33.197954 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.197886 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:18:33.280650 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280614 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a8a9eec-a25f-4626-bb15-f1447a56160b-istiod-ca-cert\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.280848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280692 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-podinfo\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.280848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280708 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-token\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.280848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280728 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-socket\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.280848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280762 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-envoy\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.280848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280801 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-certs\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.280848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280838 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8pfj\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-kube-api-access-t8pfj\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.281162 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280862 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-data\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.281162 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.280896 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-credential-socket\") pod \"9a8a9eec-a25f-4626-bb15-f1447a56160b\" (UID: \"9a8a9eec-a25f-4626-bb15-f1447a56160b\") " Jul 08 20:18:33.281162 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281093 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a8a9eec-a25f-4626-bb15-f1447a56160b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:18:33.281365 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281167 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:33.281365 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281182 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:33.281365 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281300 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:33.281498 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281367 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.281498 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281384 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a8a9eec-a25f-4626-bb15-f1447a56160b-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.281498 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.281397 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.282115 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.282092 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-data" (OuterVolumeSpecName: "istio-data") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:33.283118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.283095 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:18:33.283118 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.283097 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:18:33.283401 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.283383 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:18:33.283454 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.283402 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-kube-api-access-t8pfj" (OuterVolumeSpecName: "kube-api-access-t8pfj") pod "9a8a9eec-a25f-4626-bb15-f1447a56160b" (UID: "9a8a9eec-a25f-4626-bb15-f1447a56160b"). InnerVolumeSpecName "kube-api-access-t8pfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:18:33.382573 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.382536 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.382573 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.382567 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.382573 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.382580 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.382817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.382592 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8pfj\" (UniqueName: \"kubernetes.io/projected/9a8a9eec-a25f-4626-bb15-f1447a56160b-kube-api-access-t8pfj\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.382817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.382605 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.382817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.382617 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a8a9eec-a25f-4626-bb15-f1447a56160b-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:18:33.558542 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.558508 2569 generic.go:356] "Generic (PLEG): container finished" podID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerID="3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747" exitCode=0 Jul 08 20:18:33.558715 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.558573 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" Jul 08 20:18:33.558715 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.558606 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" event={"ID":"9a8a9eec-a25f-4626-bb15-f1447a56160b","Type":"ContainerDied","Data":"3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747"} Jul 08 20:18:33.558715 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.558670 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph" event={"ID":"9a8a9eec-a25f-4626-bb15-f1447a56160b","Type":"ContainerDied","Data":"bf74bb55fed87cce83e47b0f84fbddb96b5a85f88787227bde2171aa1603eb51"} Jul 08 20:18:33.558715 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.558706 2569 scope.go:117] "RemoveContainer" containerID="3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747" Jul 08 20:18:33.568242 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.568189 2569 scope.go:117] "RemoveContainer" containerID="2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884" Jul 08 20:18:33.575931 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.575913 2569 scope.go:117] "RemoveContainer" containerID="3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747" Jul 08 20:18:33.576195 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:18:33.576173 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747\": container with ID starting with 3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747 not found: ID does not exist" containerID="3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747" Jul 08 20:18:33.576296 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.576225 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747"} err="failed to get container status \"3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747\": rpc error: code = NotFound desc = could not find container \"3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747\": container with ID starting with 3d37ac2c6282dc05197914ffb9edac6a97f2c379f3ad94b693e17a4812127747 not found: ID does not exist" Jul 08 20:18:33.576296 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.576251 2569 scope.go:117] "RemoveContainer" containerID="2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884" Jul 08 20:18:33.576511 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:18:33.576494 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884\": container with ID starting with 2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884 not found: ID does not exist" containerID="2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884" Jul 08 20:18:33.576553 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.576515 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884"} err="failed to get container status \"2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884\": rpc error: code = NotFound desc = could not find container \"2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884\": container with ID starting with 2a94d3fb890a30386c8e9e164483bc0b01fd24a414cca6f91424faee59d96884 not found: ID does not exist" Jul 08 20:18:33.581115 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.581093 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph"] Jul 08 20:18:33.585306 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:33.585285 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb9fe-openshift-default-6fcbc5f457-pprph"] Jul 08 20:18:34.858304 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:34.858266 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" path="/var/lib/kubelet/pods/9a8a9eec-a25f-4626-bb15-f1447a56160b/volumes" Jul 08 20:18:36.649990 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.649955 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn"] Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650332 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650345 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650361 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650367 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650379 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650384 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650444 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66347b6-d407-485e-8df2-893f425b74e7" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650453 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" Jul 08 20:18:36.650504 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.650459 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a8a9eec-a25f-4626-bb15-f1447a56160b" containerName="istio-proxy" Jul 08 20:18:36.655117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.655088 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.657924 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.657901 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-27cd6d-openshift-default-dockercfg-fzvkr\"" Jul 08 20:18:36.664616 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.664588 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn"] Jul 08 20:18:36.817560 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817518 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z5z8\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-kube-api-access-6z5z8\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817574 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-data\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-envoy\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817656 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-podinfo\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817709 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-socket\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817737 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-credential-socket\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817756 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-certs\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.817817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-token\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.818057 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.817844 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istiod-ca-cert\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919270 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919126 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-data\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919270 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919181 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-envoy\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919513 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919332 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-podinfo\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919513 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919387 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-socket\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919513 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-credential-socket\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919513 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919466 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-certs\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919513 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919506 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-token\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919773 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919635 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istiod-ca-cert\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919773 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919713 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z5z8\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-kube-api-access-6z5z8\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919773 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919757 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-socket\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919945 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919828 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-credential-socket\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.919945 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919864 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-certs\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.920054 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.919984 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-data\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.920558 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.920531 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istiod-ca-cert\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.921694 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.921671 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-envoy\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.921833 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.921817 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-podinfo\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.927529 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.927508 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-token\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.927658 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.927632 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z5z8\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-kube-api-access-6z5z8\") pod \"e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:36.967807 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:36.967764 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:37.107313 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.107290 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn"] Jul 08 20:18:37.109910 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:18:37.109886 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6cc4bb_af81_43de_97f9_aecda4f203a4.slice/crio-af24dee30fd6cc447db05294b912f424b51ebf313d5091ddcea498ebcedd82a2 WatchSource:0}: Error finding container af24dee30fd6cc447db05294b912f424b51ebf313d5091ddcea498ebcedd82a2: Status 404 returned error can't find the container with id af24dee30fd6cc447db05294b912f424b51ebf313d5091ddcea498ebcedd82a2 Jul 08 20:18:37.112571 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.112541 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:37.112700 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.112608 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:37.112700 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.112638 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:37.582993 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.582958 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" event={"ID":"9b6cc4bb-af81-43de-97f9-aecda4f203a4","Type":"ContainerStarted","Data":"91c585bf87be3121bd46b9de6a132e923afd80192ec25d24f18c6938bf359684"} Jul 08 20:18:37.583234 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.583022 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" event={"ID":"9b6cc4bb-af81-43de-97f9-aecda4f203a4","Type":"ContainerStarted","Data":"af24dee30fd6cc447db05294b912f424b51ebf313d5091ddcea498ebcedd82a2"} Jul 08 20:18:37.602877 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.602822 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podStartSLOduration=1.602802428 podStartE2EDuration="1.602802428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:18:37.601611268 +0000 UTC m=+2213.258718749" watchObservedRunningTime="2026-07-08 20:18:37.602802428 +0000 UTC m=+2213.259909908" Jul 08 20:18:37.968775 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:37.968676 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:38.973415 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:38.973383 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:39.592087 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:39.592054 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:39.593226 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:39.593115 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:46.898729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:46.897798 2569 patch_prober.go:28] interesting pod/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:58120->10.132.0.47:15021: read: connection reset by peer" start-of-body= Jul 08 20:18:46.898729 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:46.897866 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:58120->10.132.0.47:15021: read: connection reset by peer" Jul 08 20:18:47.632723 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:47.632686 2569 generic.go:356] "Generic (PLEG): container finished" podID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerID="39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0" exitCode=0 Jul 08 20:18:47.632932 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:47.632763 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" event={"ID":"1f11df6a-5773-4a15-90b9-6e933c2cdebb","Type":"ContainerDied","Data":"39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0"} Jul 08 20:18:47.633219 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:47.633189 2569 scope.go:117] "RemoveContainer" containerID="39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0" Jul 08 20:18:47.634335 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:47.634307 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:47.634421 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:47.634371 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:47.634421 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:47.634400 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:48.641009 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:48.640974 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" event={"ID":"1f11df6a-5773-4a15-90b9-6e933c2cdebb","Type":"ContainerStarted","Data":"1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c"} Jul 08 20:18:49.106259 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:49.106225 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:49.106487 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:49.106463 2569 patch_prober.go:28] interesting pod/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 20:18:49.106574 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:49.106519 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 20:18:50.107051 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:50.107012 2569 patch_prober.go:28] interesting pod/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 20:18:50.107447 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:50.107080 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 20:18:51.106902 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:51.106858 2569 patch_prober.go:28] interesting pod/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 20:18:51.107094 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:51.106915 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 20:18:52.106415 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:52.106379 2569 patch_prober.go:28] interesting pod/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 08 20:18:52.106609 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:52.106443 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 08 20:18:53.110141 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:53.110111 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:53.110547 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:53.110397 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:53.111192 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:53.111177 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:18:55.634365 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:55.634320 2569 patch_prober.go:28] interesting pod/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 20:18:55.634818 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:55.634400 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 20:18:56.694681 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:56.694562 2569 generic.go:356] "Generic (PLEG): container finished" podID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerID="91c585bf87be3121bd46b9de6a132e923afd80192ec25d24f18c6938bf359684" exitCode=0 Jul 08 20:18:56.694681 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:56.694633 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" event={"ID":"9b6cc4bb-af81-43de-97f9-aecda4f203a4","Type":"ContainerDied","Data":"91c585bf87be3121bd46b9de6a132e923afd80192ec25d24f18c6938bf359684"} Jul 08 20:18:56.695134 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:56.695044 2569 scope.go:117] "RemoveContainer" containerID="91c585bf87be3121bd46b9de6a132e923afd80192ec25d24f18c6938bf359684" Jul 08 20:18:56.696059 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:56.696025 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:56.696177 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:56.696092 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:56.696177 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:56.696123 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:18:57.702314 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:57.702269 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" event={"ID":"9b6cc4bb-af81-43de-97f9-aecda4f203a4","Type":"ContainerStarted","Data":"14b7b2723314b34def24c599b33361e896e2ff652f2cfdc5ff97e1cb788b97e1"} Jul 08 20:18:57.968616 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:57.968517 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:18:57.968786 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:57.968762 2569 patch_prober.go:28] interesting pod/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 20:18:57.968847 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:57.968827 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 20:18:58.274640 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:58.274541 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz"] Jul 08 20:18:58.274817 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:58.274767 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" containerID="cri-o://1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c" gracePeriod=30 Jul 08 20:18:58.968399 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:58.968358 2569 patch_prober.go:28] interesting pod/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 20:18:58.968824 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:58.968417 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 20:18:59.968338 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:59.968299 2569 patch_prober.go:28] interesting pod/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 20:18:59.968536 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:18:59.968357 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 20:19:00.968332 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:00.968296 2569 patch_prober.go:28] interesting pod/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 08 20:19:00.968526 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:00.968355 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 08 20:19:01.972526 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:01.972487 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:19:01.972928 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:01.972757 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:19:01.973381 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:01.973361 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:19:03.526326 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.526300 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:19:03.594632 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594600 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-credential-socket\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.594851 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594658 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-podinfo\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.594851 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594680 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-data\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.594851 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594710 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-envoy\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.595030 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594889 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istiod-ca-cert\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.595030 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594945 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-socket\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.595030 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594978 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:03.595030 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.594999 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-token\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.595030 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595026 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9lk4\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-kube-api-access-k9lk4\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.595353 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595059 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-certs\") pod \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\" (UID: \"1f11df6a-5773-4a15-90b9-6e933c2cdebb\") " Jul 08 20:19:03.595353 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595253 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:03.595353 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595266 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:19:03.595498 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595476 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:03.595498 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595485 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.595601 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595507 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.595601 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595524 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.595703 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.595681 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:03.597165 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.597130 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:19:03.597375 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.597354 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-kube-api-access-k9lk4" (OuterVolumeSpecName: "kube-api-access-k9lk4") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "kube-api-access-k9lk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:19:03.597483 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.597463 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:03.597552 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.597493 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f11df6a-5773-4a15-90b9-6e933c2cdebb" (UID: "1f11df6a-5773-4a15-90b9-6e933c2cdebb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:19:03.696354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.696313 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.696354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.696348 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.696354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.696357 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.696354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.696365 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.696621 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.696373 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9lk4\" (UniqueName: \"kubernetes.io/projected/1f11df6a-5773-4a15-90b9-6e933c2cdebb-kube-api-access-k9lk4\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.696621 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.696384 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f11df6a-5773-4a15-90b9-6e933c2cdebb-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:03.730992 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.730955 2569 generic.go:356] "Generic (PLEG): container finished" podID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerID="1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c" exitCode=0 Jul 08 20:19:03.731182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.731011 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" Jul 08 20:19:03.731182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.731043 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" event={"ID":"1f11df6a-5773-4a15-90b9-6e933c2cdebb","Type":"ContainerDied","Data":"1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c"} Jul 08 20:19:03.731182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.731093 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz" event={"ID":"1f11df6a-5773-4a15-90b9-6e933c2cdebb","Type":"ContainerDied","Data":"fab4ab72e49eed7e3fd4aebc3068b3599e8f1840529a152562af295cc223193a"} Jul 08 20:19:03.731182 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.731115 2569 scope.go:117] "RemoveContainer" containerID="1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c" Jul 08 20:19:03.741416 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.741390 2569 scope.go:117] "RemoveContainer" containerID="39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0" Jul 08 20:19:03.749369 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.749347 2569 scope.go:117] "RemoveContainer" containerID="1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c" Jul 08 20:19:03.749634 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:19:03.749616 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c\": container with ID starting with 1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c not found: ID does not exist" containerID="1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c" Jul 08 20:19:03.749685 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.749642 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c"} err="failed to get container status \"1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c\": rpc error: code = NotFound desc = could not find container \"1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c\": container with ID starting with 1ee3bc662b35c6a8a29e930cb82348bbf7ca01e91b8126b2985e26ce114c8a0c not found: ID does not exist" Jul 08 20:19:03.749685 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.749660 2569 scope.go:117] "RemoveContainer" containerID="39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0" Jul 08 20:19:03.749888 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:19:03.749870 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0\": container with ID starting with 39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0 not found: ID does not exist" containerID="39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0" Jul 08 20:19:03.749965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.749895 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0"} err="failed to get container status \"39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0\": rpc error: code = NotFound desc = could not find container \"39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0\": container with ID starting with 39667e1eb15fd2ccf36be455e38558623133b7572634b311020d18e5e92165a0 not found: ID does not exist" Jul 08 20:19:03.753104 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.753079 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz"] Jul 08 20:19:03.757318 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:03.757297 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-85eb30-openshift-default-6898c9db77-r2sgz"] Jul 08 20:19:04.857849 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:04.857812 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" path="/var/lib/kubelet/pods/1f11df6a-5773-4a15-90b9-6e933c2cdebb/volumes" Jul 08 20:19:05.642217 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:05.642156 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn"] Jul 08 20:19:05.642518 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:05.642490 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" containerID="cri-o://14b7b2723314b34def24c599b33361e896e2ff652f2cfdc5ff97e1cb788b97e1" gracePeriod=30 Jul 08 20:19:06.496604 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.496563 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc"] Jul 08 20:19:06.497081 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.497058 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" Jul 08 20:19:06.497171 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.497086 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" Jul 08 20:19:06.497247 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.497236 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" Jul 08 20:19:06.502001 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.501978 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.505648 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.505628 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9caeb3-openshift-default-dockercfg-hmzcd\"" Jul 08 20:19:06.531887 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.531849 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc"] Jul 08 20:19:06.623117 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623080 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-podinfo\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623321 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623139 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xgtc\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-kube-api-access-4xgtc\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623321 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6496758d-ee19-4ad3-984f-1a98085d7a12-istiod-ca-cert\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623321 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623281 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-data\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623321 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623303 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-certs\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623321 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623317 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-envoy\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623501 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623408 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-socket\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623501 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623449 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-credential-socket\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.623501 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.623478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-token\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.724747 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724703 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xgtc\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-kube-api-access-4xgtc\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.724747 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724762 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6496758d-ee19-4ad3-984f-1a98085d7a12-istiod-ca-cert\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724792 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-data\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724821 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-certs\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724845 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-envoy\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724880 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-socket\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724910 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-credential-socket\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.724937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-token\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.725019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-podinfo\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725432 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.725238 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-data\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725432 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.725260 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-certs\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725432 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.725384 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-socket\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725558 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.725440 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-credential-socket\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.725670 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.725648 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6496758d-ee19-4ad3-984f-1a98085d7a12-istiod-ca-cert\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.727153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.727129 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-envoy\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.727595 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.727574 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-podinfo\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.736175 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.736137 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-token\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.736330 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.736305 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xgtc\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-kube-api-access-4xgtc\") pod \"e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.812849 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.812812 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:06.962181 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.962157 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc"] Jul 08 20:19:06.963820 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:19:06.963793 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6496758d_ee19_4ad3_984f_1a98085d7a12.slice/crio-cad3b8b76d321256ab11e0bbc8a3958b11e6bf60ecf06856ebce31294b2d938a WatchSource:0}: Error finding container cad3b8b76d321256ab11e0bbc8a3958b11e6bf60ecf06856ebce31294b2d938a: Status 404 returned error can't find the container with id cad3b8b76d321256ab11e0bbc8a3958b11e6bf60ecf06856ebce31294b2d938a Jul 08 20:19:06.965998 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.965966 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:06.966092 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.966043 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:06.966157 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:06.966087 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:07.757097 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:07.757060 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" event={"ID":"6496758d-ee19-4ad3-984f-1a98085d7a12","Type":"ContainerStarted","Data":"ec6225792baf92b3dd378815fcac29ad355813c13c99f9b7aa0739a04999bfde"} Jul 08 20:19:07.757524 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:07.757123 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" event={"ID":"6496758d-ee19-4ad3-984f-1a98085d7a12","Type":"ContainerStarted","Data":"cad3b8b76d321256ab11e0bbc8a3958b11e6bf60ecf06856ebce31294b2d938a"} Jul 08 20:19:07.778001 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:07.777940 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" podStartSLOduration=1.777920237 podStartE2EDuration="1.777920237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:19:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:19:07.777008845 +0000 UTC m=+2243.434116338" watchObservedRunningTime="2026-07-08 20:19:07.777920237 +0000 UTC m=+2243.435027719" Jul 08 20:19:07.813749 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:07.813710 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:08.818195 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:08.818166 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:09.767041 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:09.767009 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:09.767849 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:09.767819 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:10.773214 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.773168 2569 generic.go:356] "Generic (PLEG): container finished" podID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerID="14b7b2723314b34def24c599b33361e896e2ff652f2cfdc5ff97e1cb788b97e1" exitCode=0 Jul 08 20:19:10.773659 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.773235 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" event={"ID":"9b6cc4bb-af81-43de-97f9-aecda4f203a4","Type":"ContainerDied","Data":"14b7b2723314b34def24c599b33361e896e2ff652f2cfdc5ff97e1cb788b97e1"} Jul 08 20:19:10.773659 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.773359 2569 scope.go:117] "RemoveContainer" containerID="91c585bf87be3121bd46b9de6a132e923afd80192ec25d24f18c6938bf359684" Jul 08 20:19:10.895704 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.895676 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:19:10.961164 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961076 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-token\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961164 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961163 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-envoy\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961382 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961237 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istiod-ca-cert\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961382 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961273 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-podinfo\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961382 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961320 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-socket\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961382 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961369 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z5z8\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-kube-api-access-6z5z8\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961575 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961392 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-data\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961575 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961417 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-certs\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961575 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961456 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-credential-socket\") pod \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\" (UID: \"9b6cc4bb-af81-43de-97f9-aecda4f203a4\") " Jul 08 20:19:10.961745 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961716 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:19:10.962396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961852 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:10.962396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.961883 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:10.962396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.962248 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:10.962396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.962325 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:10.962396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.962341 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:10.963592 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.963565 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:19:10.963592 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.963579 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:19:10.963793 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.963592 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:19:10.963978 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:10.963957 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-kube-api-access-6z5z8" (OuterVolumeSpecName: "kube-api-access-6z5z8") pod "9b6cc4bb-af81-43de-97f9-aecda4f203a4" (UID: "9b6cc4bb-af81-43de-97f9-aecda4f203a4"). InnerVolumeSpecName "kube-api-access-6z5z8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:19:11.062455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062419 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z5z8\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-kube-api-access-6z5z8\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062446 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062455 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062455 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062464 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062738 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062472 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062738 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062481 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062738 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062488 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b6cc4bb-af81-43de-97f9-aecda4f203a4-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.062738 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.062496 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b6cc4bb-af81-43de-97f9-aecda4f203a4-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:19:11.780285 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.780243 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" event={"ID":"9b6cc4bb-af81-43de-97f9-aecda4f203a4","Type":"ContainerDied","Data":"af24dee30fd6cc447db05294b912f424b51ebf313d5091ddcea498ebcedd82a2"} Jul 08 20:19:11.780285 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.780273 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn" Jul 08 20:19:11.780783 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.780320 2569 scope.go:117] "RemoveContainer" containerID="14b7b2723314b34def24c599b33361e896e2ff652f2cfdc5ff97e1cb788b97e1" Jul 08 20:19:11.803080 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.803046 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn"] Jul 08 20:19:11.809982 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:11.809953 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-27cd6d-openshift-default-84fffd8c77-lhqbn"] Jul 08 20:19:12.858466 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:12.858436 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" path="/var/lib/kubelet/pods/9b6cc4bb-af81-43de-97f9-aecda4f203a4/volumes" Jul 08 20:19:14.797573 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797544 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq"] Jul 08 20:19:14.797940 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797914 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" Jul 08 20:19:14.797940 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797932 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" Jul 08 20:19:14.797940 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797949 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" Jul 08 20:19:14.797940 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797956 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" Jul 08 20:19:14.797940 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797978 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" Jul 08 20:19:14.797940 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.797988 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" Jul 08 20:19:14.798291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.798098 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" Jul 08 20:19:14.798291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.798110 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f11df6a-5773-4a15-90b9-6e933c2cdebb" containerName="istio-proxy" Jul 08 20:19:14.798291 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.798119 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6cc4bb-af81-43de-97f9-aecda4f203a4" containerName="istio-proxy" Jul 08 20:19:14.802671 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.802650 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.805396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.805370 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e06d38-openshift-default-dockercfg-v79bv\"" Jul 08 20:19:14.812769 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.812742 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq"] Jul 08 20:19:14.897586 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897543 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-certs\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897586 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897594 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-envoy\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897622 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwppw\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-kube-api-access-cwppw\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897719 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-credential-socket\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897757 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-socket\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897779 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-podinfo\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-data\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897838 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-token\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.897965 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.897865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istiod-ca-cert\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998405 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998345 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-credential-socket\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998405 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-socket\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-podinfo\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-data\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998467 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-token\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istiod-ca-cert\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-certs\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-envoy\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.998702 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998572 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwppw\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-kube-api-access-cwppw\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.999075 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998882 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-data\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.999075 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.998934 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-credential-socket\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.999185 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.999105 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-certs\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.999298 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.999221 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-socket\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:14.999357 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:14.999338 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istiod-ca-cert\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:15.000816 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.000794 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-envoy\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:15.001067 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.001044 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-podinfo\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:15.006350 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.006327 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-token\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:15.006576 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.006555 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwppw\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-kube-api-access-cwppw\") pod \"e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:15.114081 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.114045 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:15.252521 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.252495 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq"] Jul 08 20:19:15.254339 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:19:15.254310 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0231d4_007a_47d1_aa8b_5cb3023bf090.slice/crio-f6378a80d705ae378c87a18774f334dd88624ad2a685abcfd6e12e325c985309 WatchSource:0}: Error finding container f6378a80d705ae378c87a18774f334dd88624ad2a685abcfd6e12e325c985309: Status 404 returned error can't find the container with id f6378a80d705ae378c87a18774f334dd88624ad2a685abcfd6e12e325c985309 Jul 08 20:19:15.256571 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.256534 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:15.256685 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.256617 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:15.256685 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.256664 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:15.800531 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.800499 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" event={"ID":"3c0231d4-007a-47d1-aa8b-5cb3023bf090","Type":"ContainerStarted","Data":"fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b"} Jul 08 20:19:15.800962 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.800550 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" event={"ID":"3c0231d4-007a-47d1-aa8b-5cb3023bf090","Type":"ContainerStarted","Data":"f6378a80d705ae378c87a18774f334dd88624ad2a685abcfd6e12e325c985309"} Jul 08 20:19:15.825920 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:15.825862 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" podStartSLOduration=1.825843214 podStartE2EDuration="1.825843214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:19:15.82406116 +0000 UTC m=+2251.481168652" watchObservedRunningTime="2026-07-08 20:19:15.825843214 +0000 UTC m=+2251.482950694" Jul 08 20:19:16.115238 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:16.115182 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:16.120213 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:16.120170 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:16.805174 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:16.805147 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:16.806069 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:16.806047 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:23.845619 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:23.845500 2569 generic.go:356] "Generic (PLEG): container finished" podID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerID="ec6225792baf92b3dd378815fcac29ad355813c13c99f9b7aa0739a04999bfde" exitCode=0 Jul 08 20:19:23.846359 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:23.846313 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" event={"ID":"6496758d-ee19-4ad3-984f-1a98085d7a12","Type":"ContainerDied","Data":"ec6225792baf92b3dd378815fcac29ad355813c13c99f9b7aa0739a04999bfde"} Jul 08 20:19:23.847221 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:23.847174 2569 scope.go:117] "RemoveContainer" containerID="ec6225792baf92b3dd378815fcac29ad355813c13c99f9b7aa0739a04999bfde" Jul 08 20:19:23.849027 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:23.848992 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:23.849257 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:23.849232 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:23.849427 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:23.849405 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:24.767609 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:24.767573 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:24.862945 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:24.862904 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" event={"ID":"6496758d-ee19-4ad3-984f-1a98085d7a12","Type":"ContainerStarted","Data":"0b1cfcef5cf42fb9f6b558af98e99bbfad710fe216e24700b8350d7366d56027"} Jul 08 20:19:25.813647 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:25.813610 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:25.814447 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:25.814424 2569 patch_prober.go:28] interesting pod/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 20:19:25.814522 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:25.814477 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 20:19:26.813482 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:26.813440 2569 patch_prober.go:28] interesting pod/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 20:19:26.813897 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:26.813510 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 20:19:27.813708 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:27.813667 2569 patch_prober.go:28] interesting pod/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 20:19:27.814220 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:27.813774 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 20:19:28.813475 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:28.813437 2569 patch_prober.go:28] interesting pod/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 08 20:19:28.813660 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:28.813503 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 08 20:19:29.817596 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:29.817564 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:29.817978 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:29.817936 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:29.818682 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:29.818665 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:19:30.901312 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:30.901279 2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerID="fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b" exitCode=0 Jul 08 20:19:30.901779 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:30.901357 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" event={"ID":"3c0231d4-007a-47d1-aa8b-5cb3023bf090","Type":"ContainerDied","Data":"fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b"} Jul 08 20:19:30.901848 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:30.901831 2569 scope.go:117] "RemoveContainer" containerID="fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b" Jul 08 20:19:30.902871 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:30.902847 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:30.902934 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:30.902910 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:30.902971 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:30.902940 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 20:19:31.805497 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:31.805456 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:31.907436 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:31.907395 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" event={"ID":"3c0231d4-007a-47d1-aa8b-5cb3023bf090","Type":"ContainerStarted","Data":"d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3"} Jul 08 20:19:32.114717 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:32.114619 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:32.115994 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:32.115967 2569 patch_prober.go:28] interesting pod/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 08 20:19:32.116129 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:32.116038 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 08 20:19:33.115376 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:33.115335 2569 patch_prober.go:28] interesting pod/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 08 20:19:33.115773 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:33.115397 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 08 20:19:34.114476 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:34.114438 2569 patch_prober.go:28] interesting pod/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 08 20:19:34.114679 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:34.114496 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 08 20:19:35.114564 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:35.114523 2569 patch_prober.go:28] interesting pod/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 08 20:19:35.114982 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:35.114585 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 08 20:19:36.119076 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:36.119041 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:36.119472 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:36.119453 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:19:36.120123 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:19:36.120105 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:22:33.666273 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:33.666234 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc"] Jul 08 20:22:33.666770 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:33.666464 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" containerID="cri-o://0b1cfcef5cf42fb9f6b558af98e99bbfad710fe216e24700b8350d7366d56027" gracePeriod=30 Jul 08 20:22:38.802528 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.802497 2569 generic.go:356] "Generic (PLEG): container finished" podID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerID="0b1cfcef5cf42fb9f6b558af98e99bbfad710fe216e24700b8350d7366d56027" exitCode=0 Jul 08 20:22:38.803000 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.802583 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" event={"ID":"6496758d-ee19-4ad3-984f-1a98085d7a12","Type":"ContainerDied","Data":"0b1cfcef5cf42fb9f6b558af98e99bbfad710fe216e24700b8350d7366d56027"} Jul 08 20:22:38.803000 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.802650 2569 scope.go:117] "RemoveContainer" containerID="ec6225792baf92b3dd378815fcac29ad355813c13c99f9b7aa0739a04999bfde" Jul 08 20:22:38.917976 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.917948 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:22:38.937433 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937358 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-certs\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937577 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937429 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-credential-socket\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937577 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937479 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-envoy\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937577 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937512 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xgtc\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-kube-api-access-4xgtc\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937577 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937551 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-socket\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937588 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-podinfo\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937620 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6496758d-ee19-4ad3-984f-1a98085d7a12-istiod-ca-cert\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937651 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-token\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937680 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:38.937795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937691 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-data\") pod \"6496758d-ee19-4ad3-984f-1a98085d7a12\" (UID: \"6496758d-ee19-4ad3-984f-1a98085d7a12\") " Jul 08 20:22:38.937795 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937705 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:38.938093 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937907 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:38.938093 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.937995 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:38.938093 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.938013 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:38.938093 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.938027 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:38.938452 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.938418 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6496758d-ee19-4ad3-984f-1a98085d7a12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:22:38.938778 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.938751 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-data" (OuterVolumeSpecName: "istio-data") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:38.940040 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.939991 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:22:38.940357 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.940322 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:38.940689 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.940555 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-kube-api-access-4xgtc" (OuterVolumeSpecName: "kube-api-access-4xgtc") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "kube-api-access-4xgtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:22:38.941128 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:38.941102 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-token" (OuterVolumeSpecName: "istio-token") pod "6496758d-ee19-4ad3-984f-1a98085d7a12" (UID: "6496758d-ee19-4ad3-984f-1a98085d7a12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:22:39.038535 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.038495 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:39.038535 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.038528 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:39.038535 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.038541 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xgtc\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-kube-api-access-4xgtc\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:39.038799 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.038555 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:39.038799 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.038567 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6496758d-ee19-4ad3-984f-1a98085d7a12-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:39.038799 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.038581 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6496758d-ee19-4ad3-984f-1a98085d7a12-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:39.808336 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.808306 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" Jul 08 20:22:39.808336 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.808321 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc" event={"ID":"6496758d-ee19-4ad3-984f-1a98085d7a12","Type":"ContainerDied","Data":"cad3b8b76d321256ab11e0bbc8a3958b11e6bf60ecf06856ebce31294b2d938a"} Jul 08 20:22:39.808882 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.808390 2569 scope.go:117] "RemoveContainer" containerID="0b1cfcef5cf42fb9f6b558af98e99bbfad710fe216e24700b8350d7366d56027" Jul 08 20:22:39.833010 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.832979 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc"] Jul 08 20:22:39.837387 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:39.837360 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9caeb3-openshift-default-7b554c8c8c-x9kwc"] Jul 08 20:22:40.449094 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:40.449057 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq"] Jul 08 20:22:40.449370 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:40.449340 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" containerID="cri-o://d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3" gracePeriod=30 Jul 08 20:22:40.859128 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:40.859089 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" path="/var/lib/kubelet/pods/6496758d-ee19-4ad3-984f-1a98085d7a12/volumes" Jul 08 20:22:41.248264 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:41.248150 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57f7469bcd-692jn_d9831de8-1949-4c88-bb0a-88585f8442c3/authorino/0.log" Jul 08 20:22:45.669936 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.669905 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-j9m2s_00e4bccf-cb88-4671-a47c-7c0a63c7e807/manager/0.log" Jul 08 20:22:45.708812 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.708746 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:22:45.780355 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.780323 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ggxkf_ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4/manager/0.log" Jul 08 20:22:45.791453 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791427 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-socket\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791557 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791466 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-podinfo\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791557 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791492 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-data\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791557 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791514 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwppw\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-kube-api-access-cwppw\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791557 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791534 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istiod-ca-cert\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791557 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791557 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-envoy\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791766 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791586 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-certs\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791766 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791622 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-credential-socket\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.791879 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791801 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:45.791879 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791835 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-token\") pod \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\" (UID: \"3c0231d4-007a-47d1-aa8b-5cb3023bf090\") " Jul 08 20:22:45.792003 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.791971 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:45.792232 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.792134 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-certs\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.792232 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.792161 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-workload-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.792232 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.792166 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:45.792232 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.792096 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:22:45.792438 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.792367 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:45.793842 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.793811 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:22:45.794178 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.794161 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:22:45.794271 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.794192 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-kube-api-access-cwppw" (OuterVolumeSpecName: "kube-api-access-cwppw") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "kube-api-access-cwppw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:22:45.794311 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.794289 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c0231d4-007a-47d1-aa8b-5cb3023bf090" (UID: "3c0231d4-007a-47d1-aa8b-5cb3023bf090"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:22:45.842954 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.842917 2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerID="d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3" exitCode=0 Jul 08 20:22:45.843153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.842967 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" Jul 08 20:22:45.843153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.843007 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" event={"ID":"3c0231d4-007a-47d1-aa8b-5cb3023bf090","Type":"ContainerDied","Data":"d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3"} Jul 08 20:22:45.843153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.843065 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq" event={"ID":"3c0231d4-007a-47d1-aa8b-5cb3023bf090","Type":"ContainerDied","Data":"f6378a80d705ae378c87a18774f334dd88624ad2a685abcfd6e12e325c985309"} Jul 08 20:22:45.843153 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.843095 2569 scope.go:117] "RemoveContainer" containerID="d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3" Jul 08 20:22:45.855467 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.855449 2569 scope.go:117] "RemoveContainer" containerID="fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b" Jul 08 20:22:45.863423 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.863405 2569 scope.go:117] "RemoveContainer" containerID="d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3" Jul 08 20:22:45.863711 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:22:45.863693 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3\": container with ID starting with d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3 not found: ID does not exist" containerID="d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3" Jul 08 20:22:45.863761 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.863720 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3"} err="failed to get container status \"d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3\": rpc error: code = NotFound desc = could not find container \"d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3\": container with ID starting with d29219fc95c401e53c7001afade420b74f6add855333d8fa1346c62cc8dd38e3 not found: ID does not exist" Jul 08 20:22:45.863761 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.863738 2569 scope.go:117] "RemoveContainer" containerID="fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b" Jul 08 20:22:45.863987 ip-10-0-133-133 kubenswrapper[2569]: E0708 20:22:45.863969 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b\": container with ID starting with fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b not found: ID does not exist" containerID="fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b" Jul 08 20:22:45.864036 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.863995 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b"} err="failed to get container status \"fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b\": rpc error: code = NotFound desc = could not find container \"fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b\": container with ID starting with fe434655e784cefacc7b5d24569389b0f1028864a263a9298d5dbad1c8f9c62b not found: ID does not exist" Jul 08 20:22:45.867553 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.867530 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq"] Jul 08 20:22:45.870674 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.870649 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e06d38-openshift-default-7f75f4b747-z65pq"] Jul 08 20:22:45.893526 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893493 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-credential-socket\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.893526 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893520 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-token\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.893526 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893533 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-podinfo\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.893786 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893541 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-data\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.893786 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893550 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwppw\" (UniqueName: \"kubernetes.io/projected/3c0231d4-007a-47d1-aa8b-5cb3023bf090-kube-api-access-cwppw\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.893786 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893565 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istiod-ca-cert\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:45.893786 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:45.893578 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0231d4-007a-47d1-aa8b-5cb3023bf090-istio-envoy\") on node \"ip-10-0-133-133.ec2.internal\" DevicePath \"\"" Jul 08 20:22:46.139531 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:46.139505 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cp4q5_6f1a4521-25c4-4bf9-b596-5ff89de23d05/manager/0.log" Jul 08 20:22:46.246680 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:46.246652 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-4przj_cc425cb2-2580-49fa-a05a-f3f77586e1f1/copy-manifests/0.log" Jul 08 20:22:46.253682 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:46.253657 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-4przj_cc425cb2-2580-49fa-a05a-f3f77586e1f1/manager/0.log" Jul 08 20:22:46.857831 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:46.857792 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" path="/var/lib/kubelet/pods/3c0231d4-007a-47d1-aa8b-5cb3023bf090/volumes" Jul 08 20:22:47.808611 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:47.808575 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57f7469bcd-692jn_d9831de8-1949-4c88-bb0a-88585f8442c3/authorino/0.log" Jul 08 20:22:47.923437 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:47.923406 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lqj6g_f93fd591-12bf-4bfd-9126-8c3cc5af6ff7/manager/0.log" Jul 08 20:22:48.039373 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:48.039346 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6n4f9_8f489aa8-8c11-4ba8-8791-85d22b2965e8/manager/0.log" Jul 08 20:22:48.147442 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:48.147369 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5v2h6_e9a276d2-44fc-49e5-8ce4-ceda0a284f66/kuadrant-console-plugin/0.log" Jul 08 20:22:48.266736 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:48.266707 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xbf24_7e372f94-8d03-4500-9eb3-24cc7df87f7c/registry-server/0.log" Jul 08 20:22:48.399065 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:48.398985 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vnpg7_e5e68922-80af-4d06-990f-820536c8e7d0/manager/0.log" Jul 08 20:22:48.629186 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:48.629152 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cjxlt_a6a4ddd4-aa9b-4fce-900d-2a60f0045be5/manager/0.log" Jul 08 20:22:48.974672 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:48.974643 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678q97rt_26f3293a-8e17-4411-940d-48c3866be1fd/istio-proxy/0.log" Jul 08 20:22:49.313909 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:49.313881 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779dfd48cc-ztcjp_6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f/kube-auth-proxy/0.log" Jul 08 20:22:49.439015 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:49.438985 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/1.log" Jul 08 20:22:50.217815 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:50.217781 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-szdf6_d52e0c37-1875-4659-93e1-35ab4ea1b355/main/0.log" Jul 08 20:22:50.543403 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:50.543327 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-plxkk_01a7935f-5877-4bf3-aef8-fff086421fda/main/0.log" Jul 08 20:22:58.087435 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:58.087393 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rdnjw_be2ea6fc-bd9b-4078-9634-1c31078e4661/global-pull-secret-syncer/0.log" Jul 08 20:22:58.160854 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:58.160823 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-85ltm_eb8a42e5-d264-4b5b-b877-0454af519fd4/konnectivity-agent/0.log" Jul 08 20:22:58.256346 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:22:58.256316 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-133.ec2.internal_6ab0c00c572ee00d9286c0758cbfdbaf/haproxy/0.log" Jul 08 20:23:02.879398 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:02.879361 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57f7469bcd-692jn_d9831de8-1949-4c88-bb0a-88585f8442c3/authorino/0.log" Jul 08 20:23:02.915396 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:02.915369 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lqj6g_f93fd591-12bf-4bfd-9126-8c3cc5af6ff7/manager/0.log" Jul 08 20:23:02.980664 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:02.980639 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6n4f9_8f489aa8-8c11-4ba8-8791-85d22b2965e8/manager/0.log" Jul 08 20:23:03.041368 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:03.041340 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5v2h6_e9a276d2-44fc-49e5-8ce4-ceda0a284f66/kuadrant-console-plugin/0.log" Jul 08 20:23:03.110758 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:03.110725 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xbf24_7e372f94-8d03-4500-9eb3-24cc7df87f7c/registry-server/0.log" Jul 08 20:23:03.212808 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:03.212721 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vnpg7_e5e68922-80af-4d06-990f-820536c8e7d0/manager/0.log" Jul 08 20:23:03.312877 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:03.312824 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cjxlt_a6a4ddd4-aa9b-4fce-900d-2a60f0045be5/manager/0.log" Jul 08 20:23:05.364438 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:05.364411 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jwrp8_e8dbc954-b59e-4665-a01a-e99be7e1bae6/node-exporter/0.log" Jul 08 20:23:05.385143 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:05.385114 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jwrp8_e8dbc954-b59e-4665-a01a-e99be7e1bae6/kube-rbac-proxy/0.log" Jul 08 20:23:05.405927 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:05.405906 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jwrp8_e8dbc954-b59e-4665-a01a-e99be7e1bae6/init-textfile/0.log" Jul 08 20:23:07.236837 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:07.236780 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-bfbrm_311603ab-67b8-4d51-92b9-2522a8e68cb3/networking-console-plugin/0.log" Jul 08 20:23:08.055320 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055289 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9"] Jul 08 20:23:08.055624 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055612 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" Jul 08 20:23:08.055672 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055626 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" Jul 08 20:23:08.055672 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055638 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" Jul 08 20:23:08.055672 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055644 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" Jul 08 20:23:08.055672 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055654 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" Jul 08 20:23:08.055672 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055660 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" Jul 08 20:23:08.055845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055677 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" Jul 08 20:23:08.055845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055682 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" Jul 08 20:23:08.055845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055734 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" Jul 08 20:23:08.055845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055741 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0231d4-007a-47d1-aa8b-5cb3023bf090" containerName="istio-proxy" Jul 08 20:23:08.055845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055747 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" Jul 08 20:23:08.055845 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.055751 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6496758d-ee19-4ad3-984f-1a98085d7a12" containerName="istio-proxy" Jul 08 20:23:08.060046 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.060027 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.063080 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.063034 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-np6j7\"/\"default-dockercfg-q8fwp\"" Jul 08 20:23:08.064402 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.064386 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-np6j7\"/\"kube-root-ca.crt\"" Jul 08 20:23:08.064500 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.064420 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-np6j7\"/\"openshift-service-ca.crt\"" Jul 08 20:23:08.068550 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.068527 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9"] Jul 08 20:23:08.188418 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.188376 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-proc\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.188418 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.188420 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-lib-modules\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.188656 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.188440 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-podres\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.188656 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.188550 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nffl5\" (UniqueName: \"kubernetes.io/projected/f0d32bb3-1a37-449d-bd81-70f748a456a7-kube-api-access-nffl5\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.188656 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.188599 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-sys\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289133 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-proc\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289133 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289136 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-lib-modules\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289157 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-podres\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289231 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nffl5\" (UniqueName: \"kubernetes.io/projected/f0d32bb3-1a37-449d-bd81-70f748a456a7-kube-api-access-nffl5\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289265 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-sys\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289283 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-proc\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289328 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-lib-modules\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289345 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-sys\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.289620 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.289385 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0d32bb3-1a37-449d-bd81-70f748a456a7-podres\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.297021 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.297000 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nffl5\" (UniqueName: \"kubernetes.io/projected/f0d32bb3-1a37-449d-bd81-70f748a456a7-kube-api-access-nffl5\") pod \"perf-node-gather-daemonset-275d9\" (UID: \"f0d32bb3-1a37-449d-bd81-70f748a456a7\") " pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.371836 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.371743 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.505367 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.505095 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9"] Jul 08 20:23:08.508557 ip-10-0-133-133 kubenswrapper[2569]: W0708 20:23:08.508512 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0d32bb3_1a37_449d_bd81_70f748a456a7.slice/crio-1370568b2628c37fa79c59e98aee8e6da230f3191963d1d9a87f7cfb6e655c0c WatchSource:0}: Error finding container 1370568b2628c37fa79c59e98aee8e6da230f3191963d1d9a87f7cfb6e655c0c: Status 404 returned error can't find the container with id 1370568b2628c37fa79c59e98aee8e6da230f3191963d1d9a87f7cfb6e655c0c Jul 08 20:23:08.948354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.948261 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" event={"ID":"f0d32bb3-1a37-449d-bd81-70f748a456a7","Type":"ContainerStarted","Data":"a4cb44b396bf192cb1086d8512b8795d14984a97df23841b0ea6ba1323e96a14"} Jul 08 20:23:08.948354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.948317 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" event={"ID":"f0d32bb3-1a37-449d-bd81-70f748a456a7","Type":"ContainerStarted","Data":"1370568b2628c37fa79c59e98aee8e6da230f3191963d1d9a87f7cfb6e655c0c"} Jul 08 20:23:08.948354 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.948348 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:08.965009 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:08.964961 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" podStartSLOduration=0.964945672 podStartE2EDuration="964.945672ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 20:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:23:08.963695558 +0000 UTC m=+2484.620803040" watchObservedRunningTime="2026-07-08 20:23:08.964945672 +0000 UTC m=+2484.622053165" Jul 08 20:23:10.258511 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:10.258476 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r88l4_ec911e95-6f22-4710-9bf4-d3ebf667c684/dns/0.log" Jul 08 20:23:10.281411 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:10.281380 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r88l4_ec911e95-6f22-4710-9bf4-d3ebf667c684/kube-rbac-proxy/0.log" Jul 08 20:23:10.307322 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:10.307295 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6phpd_3e17ecd3-1b9f-4cc2-8df5-d6ba168c3e84/dns-node-resolver/0.log" Jul 08 20:23:10.941301 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:10.941274 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ppvcz_9a05fefd-7dcf-4fdc-833e-6ee404a4de0f/node-ca/0.log" Jul 08 20:23:11.869939 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:11.869896 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678q97rt_26f3293a-8e17-4411-940d-48c3866be1fd/istio-proxy/0.log" Jul 08 20:23:12.014344 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:12.014315 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779dfd48cc-ztcjp_6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f/kube-auth-proxy/0.log" Jul 08 20:23:12.044225 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:12.044177 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/0.log" Jul 08 20:23:12.080093 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:12.080053 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/1.log" Jul 08 20:23:12.837039 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:12.837003 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-24rzx_c24c75a7-a8f5-4534-85e6-8f9431128780/serve-healthcheck-canary/0.log" Jul 08 20:23:13.410008 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:13.409968 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sgkx5_aea20af8-ca7e-4d2f-af08-6ec4287357ad/kube-rbac-proxy/0.log" Jul 08 20:23:13.432261 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:13.432234 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sgkx5_aea20af8-ca7e-4d2f-af08-6ec4287357ad/exporter/0.log" Jul 08 20:23:13.453246 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:13.453215 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sgkx5_aea20af8-ca7e-4d2f-af08-6ec4287357ad/extractor/0.log" Jul 08 20:23:14.963184 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:14.963152 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-np6j7/perf-node-gather-daemonset-275d9" Jul 08 20:23:15.567183 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:15.567154 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-j9m2s_00e4bccf-cb88-4671-a47c-7c0a63c7e807/manager/0.log" Jul 08 20:23:15.675527 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:15.675494 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ggxkf_ad9fa3d9-3950-4e8d-8c3f-ca3e9f0671f4/manager/0.log" Jul 08 20:23:15.839472 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:15.839373 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cp4q5_6f1a4521-25c4-4bf9-b596-5ff89de23d05/manager/0.log" Jul 08 20:23:15.861626 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:15.861595 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-4przj_cc425cb2-2580-49fa-a05a-f3f77586e1f1/manager/0.log" Jul 08 20:23:15.883112 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:15.883086 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795d769c5f-4przj_cc425cb2-2580-49fa-a05a-f3f77586e1f1/copy-manifests/0.log" Jul 08 20:23:18.358342 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:18.358315 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678q97rt_26f3293a-8e17-4411-940d-48c3866be1fd/istio-proxy/0.log" Jul 08 20:23:18.471535 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:18.471503 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779dfd48cc-ztcjp_6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f/kube-auth-proxy/0.log" Jul 08 20:23:18.488880 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:18.488853 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/0.log" Jul 08 20:23:18.520160 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:18.520128 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/1.log" Jul 08 20:23:19.746909 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:19.746876 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678q97rt_26f3293a-8e17-4411-940d-48c3866be1fd/istio-proxy/0.log" Jul 08 20:23:19.869853 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:19.869821 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779dfd48cc-ztcjp_6961e0d3-088a-4bc9-8ab7-c4e1f9f7c25f/kube-auth-proxy/0.log" Jul 08 20:23:19.894217 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:19.894131 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/0.log" Jul 08 20:23:19.925155 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:19.925125 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ftnnl_ffc6cea3-48a3-4f5f-87d5-ac7e7f97c74b/istio-proxy/1.log" Jul 08 20:23:24.081950 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.081923 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/kube-multus-additional-cni-plugins/0.log" Jul 08 20:23:24.102945 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.102915 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/egress-router-binary-copy/0.log" Jul 08 20:23:24.122370 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.122291 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/cni-plugins/0.log" Jul 08 20:23:24.142931 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.142903 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/bond-cni-plugin/0.log" Jul 08 20:23:24.163321 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.163298 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/routeoverride-cni/0.log" Jul 08 20:23:24.184009 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.183985 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/whereabouts-cni-bincopy/0.log" Jul 08 20:23:24.204635 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.204611 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdhtx_9ccc8460-155d-4388-8080-8d3c458aa401/whereabouts-cni/0.log" Jul 08 20:23:24.301645 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.301613 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x6qqv_9444ef70-a6fa-4055-9643-60bff9cac87b/kube-multus/0.log" Jul 08 20:23:24.321248 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.321222 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dvcrj_42b16c47-efdc-4d44-9501-99711b3e6487/network-metrics-daemon/0.log" Jul 08 20:23:24.340356 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:24.340331 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dvcrj_42b16c47-efdc-4d44-9501-99711b3e6487/kube-rbac-proxy/0.log" Jul 08 20:23:25.260675 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.260645 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/ovn-controller/0.log" Jul 08 20:23:25.278496 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.278466 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/ovn-acl-logging/0.log" Jul 08 20:23:25.308690 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.308658 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/kube-rbac-proxy-node/0.log" Jul 08 20:23:25.333256 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.333224 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 20:23:25.349465 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.349434 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/northd/0.log" Jul 08 20:23:25.370540 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.370508 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/nbdb/0.log" Jul 08 20:23:25.391660 ip-10-0-133-133 kubenswrapper[2569]: I0708 20:23:25.391625 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92tws_39a4d2a3-681f-4b52-ba42-7f32b11189bb/sbdb/0.log"