Jul 12 08:06:55.444006 ip-10-0-139-48 systemd[1]: Starting Kubernetes Kubelet... Jul 12 08:06:55.875490 ip-10-0-139-48 systemd[1]: Started Kubernetes Kubelet. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.799012 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804177 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804189 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804195 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804199 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804202 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804206 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:06:55.929204 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804208 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804211 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804214 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804217 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804219 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804222 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804224 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804227 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804230 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804233 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804236 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804239 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804242 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804244 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804247 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804249 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804252 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804254 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804257 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804260 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:06:55.930262 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804262 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804265 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804269 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804272 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804274 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804277 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804280 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804283 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804286 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804289 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804291 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804294 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804296 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804299 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804301 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804304 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804306 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804309 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804312 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804314 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:06:55.932908 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804318 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804320 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804323 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804326 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804328 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804331 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804333 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804336 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804339 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804341 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804344 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804346 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804350 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804354 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804359 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804362 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804365 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804368 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804370 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:06:55.933672 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804373 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804375 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804379 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804383 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804385 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804388 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804390 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804393 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804396 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804398 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804401 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804404 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804407 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804409 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804412 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804415 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804444 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804448 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804451 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804454 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:06:55.934467 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804457 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804460 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804463 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804466 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804469 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804472 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804475 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804478 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804482 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804485 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804488 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804491 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804493 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.804496 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805275 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805284 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805288 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805291 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805294 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:06:55.935178 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805297 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805300 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805302 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805306 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805309 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805311 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805314 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805316 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805319 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805322 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805324 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805327 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805329 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805333 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805337 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805341 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805345 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805348 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805351 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:06:55.937486 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805355 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805359 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805362 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805368 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805371 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805374 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805377 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805379 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805382 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805386 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805389 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805392 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805395 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805398 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805401 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805403 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805406 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805408 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805411 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805413 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:06:55.938253 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805417 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805419 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805422 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805424 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805428 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805430 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805433 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805435 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805438 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805440 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805443 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805445 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805448 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805450 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805453 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805455 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805461 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805464 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805467 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:06:55.939817 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805469 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805471 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805474 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805479 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805482 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805484 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805487 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805490 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805492 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805494 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805497 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805500 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805503 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805505 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805508 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805510 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805513 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805516 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805518 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805521 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:06:55.940558 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805523 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805525 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805528 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805531 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805533 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805535 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805538 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805540 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805543 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805545 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805554 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805557 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805559 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805561 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805564 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.805571 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806052 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806058 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806061 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806065 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:06:55.942134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806067 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806070 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806073 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806076 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806079 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806082 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806084 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806087 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806089 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806092 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806095 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806098 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806100 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806103 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806106 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806110 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806113 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806115 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806118 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806121 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:06:55.942904 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806123 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806126 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806134 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806137 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806139 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806142 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806144 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806149 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806151 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806154 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806156 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806159 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806162 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806164 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806167 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806169 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806172 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806175 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806177 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806179 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:06:55.944403 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806182 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806184 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806187 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806190 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806192 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806195 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806197 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806200 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806202 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806205 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806207 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806210 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806213 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806215 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806218 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806227 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806231 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806234 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806239 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806243 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:06:55.945181 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806245 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806248 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806250 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806253 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806256 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806258 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806261 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806263 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806266 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806268 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806271 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806275 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806278 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806281 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806285 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806288 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806290 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806293 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806296 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:06:55.946733 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806299 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806301 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806304 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806307 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806309 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806312 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806315 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806318 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806322 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806371 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806596 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806607 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806614 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806619 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806624 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.806629 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806792 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806804 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806823 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806830 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806836 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 08:06:55.947488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806842 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806848 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806854 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806865 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806870 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806876 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806881 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806886 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806891 2558 flags.go:64] FLAG: --cgroup-root="" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806896 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806900 2558 flags.go:64] FLAG: --client-ca-file="" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806905 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806915 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806942 2558 flags.go:64] FLAG: --cluster-domain="" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806947 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806953 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806958 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806964 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806970 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806975 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.806985 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807008 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807015 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807023 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 08:06:55.948653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807028 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807033 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807038 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807045 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807055 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807060 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807064 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807069 2558 flags.go:64] FLAG: --enable-server="true" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807074 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807089 2558 flags.go:64] FLAG: --event-burst="100" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807095 2558 flags.go:64] FLAG: --event-qps="50" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807099 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807104 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807113 2558 flags.go:64] FLAG: --eviction-hard="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807119 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807124 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807129 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807134 2558 flags.go:64] FLAG: --eviction-soft="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807139 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807144 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807148 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807159 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807163 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807168 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807173 2558 flags.go:64] FLAG: --feature-gates="" Jul 12 08:06:55.949442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807179 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807184 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807189 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807194 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807198 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807294 2558 flags.go:64] FLAG: --help="false" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807433 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-139-48.ec2.internal" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807443 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807448 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807452 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807456 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807460 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807463 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807466 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807469 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807472 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807475 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807479 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807482 2558 flags.go:64] FLAG: --kube-reserved="" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807485 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807488 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807492 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807495 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807498 2558 flags.go:64] FLAG: --lock-file="" Jul 12 08:06:55.950210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807501 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807504 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807508 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807522 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807525 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807528 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807531 2558 flags.go:64] FLAG: --logging-format="text" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807534 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807538 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807541 2558 flags.go:64] FLAG: --manifest-url="" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807544 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807553 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807556 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807560 2558 flags.go:64] FLAG: --max-pods="110" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807563 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807567 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807570 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807573 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807577 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807579 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807582 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807592 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807595 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807598 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 08:06:55.950957 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807601 2558 flags.go:64] FLAG: --pod-cidr="" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807604 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807610 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807613 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807617 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807620 2558 flags.go:64] FLAG: --port="10250" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807623 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807626 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08b44482cc84a9726" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807629 2558 flags.go:64] FLAG: --qos-reserved="" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807633 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807636 2558 flags.go:64] FLAG: --register-node="true" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807639 2558 flags.go:64] FLAG: --register-with-taints="" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807643 2558 flags.go:64] FLAG: --registry-burst="10" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807646 2558 flags.go:64] FLAG: --registry-qps="5" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807649 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807652 2558 flags.go:64] FLAG: --reserved-memory="" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807656 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807659 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807662 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807666 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807670 2558 flags.go:64] FLAG: --runonce="false" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807673 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807676 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807679 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807682 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 08:06:55.951730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807686 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807689 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807693 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807696 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807699 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807702 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807704 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807707 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807710 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807713 2558 flags.go:64] FLAG: --system-cgroups="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807716 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807723 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807726 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807729 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807734 2558 flags.go:64] FLAG: --tls-min-version="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807738 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807796 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807800 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807805 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807808 2558 flags.go:64] FLAG: --v="2" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807813 2558 flags.go:64] FLAG: --version="false" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807820 2558 flags.go:64] FLAG: --vmodule="" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807824 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.807827 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807951 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:06:55.952498 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807956 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807960 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807963 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807966 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807969 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807972 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807974 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807977 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807980 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807983 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807986 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807988 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807995 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.807998 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808001 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808003 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808006 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808009 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808014 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:06:55.953265 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808018 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808021 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808024 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808027 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808037 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808040 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808043 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808046 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808049 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808051 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808054 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808057 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808060 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808062 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808066 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808069 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808071 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808074 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808076 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808079 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:06:55.954043 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808081 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808084 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808086 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808091 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808094 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808098 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808100 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808103 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808105 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808108 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808111 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808113 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808116 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808118 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808121 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808123 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808126 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808136 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808139 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808142 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:06:55.957903 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808144 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808147 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808151 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808155 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808159 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808162 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808165 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808168 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808171 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808173 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808176 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808179 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808181 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808184 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808186 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808190 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808193 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808197 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808199 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:06:55.958655 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808202 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808204 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808207 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808210 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808212 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808215 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808218 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808221 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808224 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808226 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808229 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808237 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808240 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808243 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808245 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808248 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808250 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808253 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808255 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:06:55.959409 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.808258 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.809184 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.815903 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.815919 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.815932 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.815938 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.815996 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816001 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816004 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816007 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816010 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816014 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816016 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816019 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:06:55.960970 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816022 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816025 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816028 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816030 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816033 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816036 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816039 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816042 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816045 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816048 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816056 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816059 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816062 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816065 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816067 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816070 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816072 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816075 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816077 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816080 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:06:55.961734 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816083 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816086 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816088 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816091 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816094 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816097 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816099 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816102 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816106 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816109 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816112 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816115 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816118 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816121 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816123 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816126 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816128 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816131 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816134 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:06:55.962515 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816136 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816139 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816141 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816145 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816150 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816154 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816158 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816160 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816163 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816166 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816168 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816171 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816174 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816179 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816182 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816185 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816187 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816190 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816192 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:06:55.963670 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816195 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816198 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816200 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816203 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816206 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816208 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816211 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816214 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816216 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816218 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816221 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816224 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816226 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816229 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816231 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816234 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816236 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816239 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816242 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816245 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:06:55.964436 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816248 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816250 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816253 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816256 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816258 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816261 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816265 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816267 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816270 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816272 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816275 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816277 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816280 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.816285 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816411 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:06:56.058250 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816415 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816418 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816422 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816425 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816427 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816430 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816432 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816434 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816437 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816440 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816442 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816445 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816447 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816450 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816452 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816455 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816458 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816460 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816463 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816466 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:06:56.058988 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816469 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816472 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816474 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816477 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816481 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816483 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816486 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816488 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816491 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816494 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816496 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816499 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816501 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816504 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816507 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816511 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816514 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816516 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816519 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:06:56.060141 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816522 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816525 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816527 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816529 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816533 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816537 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816540 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816543 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816546 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816549 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816553 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816555 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816558 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816561 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816564 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816566 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816569 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816573 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816576 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816578 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:06:56.061306 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816581 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816584 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816586 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816589 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816591 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816595 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816599 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816602 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816604 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816607 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816610 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816613 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816615 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816618 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816621 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816624 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816628 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816631 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816633 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:06:56.062878 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816636 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816638 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816641 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816643 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816646 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816649 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816651 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816653 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816656 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816658 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816662 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816664 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816667 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816669 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816672 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816674 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816677 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816679 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816682 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:06:56.063634 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:55.816684 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.816689 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.816805 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.819454 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.820689 2558 server.go:1019] "Starting client certificate rotation" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.820792 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.820844 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.822839 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.824316 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.838159 2558 log.go:25] "Validated CRI v1 runtime API" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.840920 2558 log.go:25] "Validated CRI v1 image API" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.841644 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 08:06:56.064827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.843725 2558 fs.go:135] Filesystem UUIDs: map[14ccc84d-9bf1-4e35-b398-34595072e4ee:/dev/nvme0n1p4 4f12ff20-ad5b-4e87-93ef-d9fdeac54f7c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 12 08:06:56.065193 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.843753 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 08:06:56.065193 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.846074 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.852951 2558 manager.go:216] Machine: {Timestamp:2026-07-12 08:06:55.851783762 +0000 UTC m=+0.297268886 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100067 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2058889ddcb2643afdfc632cbde3c9 SystemUUID:ec205888-9ddc-b264-3afd-fc632cbde3c9 BootID:b44fe239-85e7-460e-be53-657a7762fd0b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:59:88:e0:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:59:88:e0:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:d2:75:36:20:21 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.853044 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.853116 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.855970 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.856011 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-48.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.856665 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.856674 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.856682 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.857661 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.859116 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.864270 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jvcm2" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866273 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866385 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866413 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866425 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866434 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 08:06:56.066766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866443 2558 kubelet.go:398] "Adding apiserver pod source" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866450 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866835 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.866853 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.868860 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.870789 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.870816 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871007 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871018 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871024 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871029 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871038 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871043 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871402 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871408 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871414 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871420 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871428 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871435 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871457 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.871463 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.873507 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jvcm2" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.874604 2558 server.go:1268] "Started kubelet" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.874721 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.874817 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.874863 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.876726 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.878529 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.882698 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.882778 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:55.882852 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.883324 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.883319 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.883345 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.883410 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 12 08:06:56.068298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:55.883418 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 12 08:06:56.069579 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.068845 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.069579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.069486 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 08:06:56.069579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.069534 2558 factory.go:55] Registering systemd factory Jul 12 08:06:56.069579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.069582 2558 factory.go:223] Registration of the systemd container factory successfully Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.070104 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-48.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.070284 2558 factory.go:153] Registering CRI-O factory Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.070303 2558 factory.go:223] Registration of the crio container factory successfully Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.070330 2558 factory.go:103] Registering Raw factory Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.070378 2558 manager.go:1196] Started watching for new ooms in manager Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.071012 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.071015 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.071101 2558 manager.go:349] Starting recovery of all containers Jul 12 08:06:56.073668 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.071124 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 08:06:56.077832 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.075449 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:06:56.077832 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.075802 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 08:06:56.083044 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.079522 2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-48.ec2.internal\" not found" node="ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.086572 2558 manager.go:354] Recovery completed Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.091103 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.092849 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.092874 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.092885 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.093396 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.093408 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.093418 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.093426 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.095582 2558 policy_none.go:49] "None policy: Start" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.095594 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.095603 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.097549 2558 policy_none.go:47] "Start" Jul 12 08:06:56.189404 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.169340 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.307925 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.215205 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.307925 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.215233 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.307925 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.215245 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.307925 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.270021 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.326894 2558 manager.go:337] "Starting Device Plugin manager" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.327051 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.327063 2558 server.go:87] "Starting device plugin registration server" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.327278 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.327290 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.327407 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.327502 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.327512 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.327954 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.328004 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.381164 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.381202 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.381225 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.381260 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.383470 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.428332 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.429239 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.429275 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.429291 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.429320 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.439141 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.439161 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-48.ec2.internal\": node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.453923 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.482318 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal"] Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.482375 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.483273 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.483297 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.483307 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485115 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485251 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485280 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485766 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485796 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485814 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485795 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485890 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.486522 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.485901 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.494147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.488413 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.494147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.488436 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:06:56.494147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.489344 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:06:56.494147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.489369 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:06:56.494147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.489379 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:06:56.610694 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.553953 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.610694 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.571897 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/adf1dd458608a3497f114fbce5c49409-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal\" (UID: \"adf1dd458608a3497f114fbce5c49409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.610694 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.571917 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/adf1dd458608a3497f114fbce5c49409-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal\" (UID: \"adf1dd458608a3497f114fbce5c49409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.719740 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.655049 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.719740 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.672687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/adf1dd458608a3497f114fbce5c49409-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal\" (UID: \"adf1dd458608a3497f114fbce5c49409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.719740 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.672708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/adf1dd458608a3497f114fbce5c49409-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal\" (UID: \"adf1dd458608a3497f114fbce5c49409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.719740 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.672727 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/477ee7992b12ed0719fbc7684ecfecc6-config\") pod \"kube-apiserver-proxy-ip-10-0-139-48.ec2.internal\" (UID: \"477ee7992b12ed0719fbc7684ecfecc6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.719740 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.672784 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/adf1dd458608a3497f114fbce5c49409-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal\" (UID: \"adf1dd458608a3497f114fbce5c49409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.719740 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.672808 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/adf1dd458608a3497f114fbce5c49409-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal\" (UID: \"adf1dd458608a3497f114fbce5c49409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.756041 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.756022 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.773247 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.773222 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/477ee7992b12ed0719fbc7684ecfecc6-config\") pod \"kube-apiserver-proxy-ip-10-0-139-48.ec2.internal\" (UID: \"477ee7992b12ed0719fbc7684ecfecc6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.773312 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.773270 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/477ee7992b12ed0719fbc7684ecfecc6-config\") pod \"kube-apiserver-proxy-ip-10-0-139-48.ec2.internal\" (UID: \"477ee7992b12ed0719fbc7684ecfecc6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.774758 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.774743 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-48.ec2.internal\" not found" node="ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.776440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.776425 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.779430 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.779412 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-48.ec2.internal\" not found" node="ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.782978 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.782367 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" Jul 12 08:06:56.819752 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.819723 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 08:06:56.819912 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.819889 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 08:06:56.819965 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.819888 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 08:06:56.857842 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.857802 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:56.882550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.882482 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 08:01:56 +0000 UTC" deadline="2028-02-13 02:19:52.172166756 +0000 UTC" Jul 12 08:06:56.882550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:56.882519 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13938h12m55.289651093s" Jul 12 08:06:56.963952 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:56.963927 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:57.070138 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.070101 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:57.171103 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.171038 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:57.271602 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.271579 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:57.372480 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.372449 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:57.469889 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:57.469798 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477ee7992b12ed0719fbc7684ecfecc6.slice/crio-ef296ba05210d35d81f6f3cf18a5fe7ca237aa0908f649b82cf6e6edba1f6fc3 WatchSource:0}: Error finding container ef296ba05210d35d81f6f3cf18a5fe7ca237aa0908f649b82cf6e6edba1f6fc3: Status 404 returned error can't find the container with id ef296ba05210d35d81f6f3cf18a5fe7ca237aa0908f649b82cf6e6edba1f6fc3 Jul 12 08:06:57.472895 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.472877 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-48.ec2.internal\" not found" Jul 12 08:06:57.477675 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.477658 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:06:57.484563 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:57.484541 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf1dd458608a3497f114fbce5c49409.slice/crio-8163f1a84026091650647889b116a57623eee885391875c97779d50320fb7e8a WatchSource:0}: Error finding container 8163f1a84026091650647889b116a57623eee885391875c97779d50320fb7e8a: Status 404 returned error can't find the container with id 8163f1a84026091650647889b116a57623eee885391875c97779d50320fb7e8a Jul 12 08:06:57.523694 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.523675 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:06:57.583822 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.583794 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" Jul 12 08:06:57.591506 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.591481 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:06:57.595670 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.595650 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 08:06:57.596852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.596836 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" Jul 12 08:06:57.605634 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.605618 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 08:06:57.811141 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.811117 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:06:57.867142 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.867117 2558 apiserver.go:52] "Watching apiserver" Jul 12 08:06:57.872951 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.872930 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 08:06:57.873331 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.873308 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-gjm7q","openshift-dns/node-resolver-zgdp2","openshift-multus/multus-additional-cni-plugins-29ggz","openshift-multus/multus-xb5hm","openshift-network-diagnostics/network-check-target-xsdxv","openshift-network-operator/iptables-alerter-d268w","openshift-ovn-kubernetes/ovnkube-node-chzp5","kube-system/global-pull-secret-syncer-8gf2t","kube-system/konnectivity-agent-4gg28","kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz","openshift-cluster-node-tuning-operator/tuned-z5tzl","openshift-image-registry/node-ca-lnw96","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal"] Jul 12 08:06:57.876271 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.876252 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.878476 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.878457 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:57.878834 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.878810 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 08:06:57.878969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.878943 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.879087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.879007 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.879087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.879034 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 08:06:57.879213 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.879007 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 08:06:57.879213 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.879101 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 08:06:57.879213 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.879103 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ngfpj\"" Jul 12 08:06:57.880550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.880529 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.880833 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.880815 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 08:06:57.880926 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.880903 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.880990 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.880927 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.881283 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.881037 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6h94b\"" Jul 12 08:06:57.882938 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.882919 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.882938 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.882932 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.883087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.882932 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 08:06:57.883259 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.883237 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 08:06:57.883357 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.883321 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 08:06:57.883420 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.883366 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vgbsg\"" Jul 12 08:06:57.883474 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.883431 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.884144 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.883676 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 08:06:57.885421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.885394 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rnq6p\"" Jul 12 08:06:57.885530 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.885515 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 08:06:57.885850 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.885833 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:06:57.885923 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.885906 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:06:57.888425 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.888410 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:57.890345 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.890328 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.890892 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.890785 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.890892 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.890795 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 08:06:57.891028 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.890863 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xx94t\"" Jul 12 08:06:57.892110 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.892087 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:57.894514 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.894479 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 08:06:57.894514 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.894503 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 08:06:57.894668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.894557 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vvfz4\"" Jul 12 08:06:57.895150 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.895128 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:57.895234 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.895209 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:06:57.896527 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.896507 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 08:06:57.897651 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.897632 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:57.899744 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.899725 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.899841 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.899813 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q5qlv\"" Jul 12 08:06:57.899903 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.899842 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.900217 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.900202 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:57.900291 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:57.900269 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:06:57.902608 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.902591 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.904666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.904649 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 08:06:57.904846 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.904830 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.904906 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.904845 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9lbgh\"" Jul 12 08:06:57.905124 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.905108 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.905178 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.905128 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:57.907254 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.907238 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 08:06:57.907319 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.907278 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:06:57.907371 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.907347 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f2w5d\"" Jul 12 08:06:57.923857 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.923839 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f4j5c" Jul 12 08:06:57.931152 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.931137 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f4j5c" Jul 12 08:06:57.983634 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983610 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983648 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldh7n\" (UniqueName: \"kubernetes.io/projected/3a49d418-33d8-4566-aed2-dad75f7401bb-kube-api-access-ldh7n\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983676 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-registration-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983792 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-slash\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983827 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-cni-bin\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983879 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-ovn\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983904 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-socket-dir-parent\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983929 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-etc-kubernetes\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.983969 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-socket-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984004 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984038 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-cni-bin\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.984130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984101 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-cni-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.984563 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-cnibin\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.984563 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984175 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7600187-1b9f-4f03-bf5b-6c6d91140376-cni-binary-copy\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.984563 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984197 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpzbq\" (UniqueName: \"kubernetes.io/projected/47fe965a-1957-408e-8cf6-725c09aeccbc-kube-api-access-xpzbq\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.984706 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984537 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.984977 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984809 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.984977 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a49d418-33d8-4566-aed2-dad75f7401bb-tmp-dir\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:57.984977 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.984924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-device-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985487 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-sys-fs\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985675 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-kubelet\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-systemd\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985741 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985791 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cnibin\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985842 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-log-socket\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985875 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-cni-netd\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-hostroot\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.985992 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-etc-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986035 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47fe965a-1957-408e-8cf6-725c09aeccbc-ovn-node-metrics-cert\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986067 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/902ddd8c-1f3c-423f-bb33-498fcd687380-host\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986096 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-k8s-cni-cncf-io\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986129 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-systemd-units\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986159 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-kubelet\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986190 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-conf-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.986230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986230 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/bfe7c78e-0e82-4e62-8c41-0585a45658b5-kube-api-access-55tg9\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986284 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-multus-certs\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986325 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986358 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986402 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-env-overrides\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986434 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-daemon-config\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986481 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxwkm\" (UniqueName: \"kubernetes.io/projected/e7600187-1b9f-4f03-bf5b-6c6d91140376-kube-api-access-lxwkm\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986515 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-var-lib-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-node-log\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986599 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-ovnkube-script-lib\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-system-cni-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986721 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-os-release\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986756 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-cni-multus\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986788 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-modprobe-d\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986832 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-ovnkube-config\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-run-netns\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:57.987172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.986976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/902ddd8c-1f3c-423f-bb33-498fcd687380-serviceca\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987011 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spv7r\" (UniqueName: \"kubernetes.io/projected/902ddd8c-1f3c-423f-bb33-498fcd687380-kube-api-access-spv7r\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987070 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-os-release\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987161 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t22lr\" (UniqueName: \"kubernetes.io/projected/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-kube-api-access-t22lr\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987193 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-netns\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987255 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a49d418-33d8-4566-aed2-dad75f7401bb-hosts-file\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987292 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-system-cni-dir\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:57.987930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:57.987379 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.087852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087776 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-env-overrides\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.087852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087814 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-daemon-config\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.087852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087837 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxwkm\" (UniqueName: \"kubernetes.io/projected/e7600187-1b9f-4f03-bf5b-6c6d91140376-kube-api-access-lxwkm\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087865 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-systemd\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087890 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fdee555-6f54-463e-aa51-aca5ba8ba135-tmp\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087914 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-var-lib-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087937 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-node-log\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087962 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-ovnkube-script-lib\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.087992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-system-cni-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088018 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-os-release\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-cni-multus\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-modprobe-d\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.088112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088091 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-sys\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088119 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-ovnkube-config\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088166 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysconfig\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088206 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088248 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-run-netns\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088285 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/902ddd8c-1f3c-423f-bb33-498fcd687380-serviceca\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088354 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-os-release\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088357 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-modprobe-d\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088403 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-cni-multus\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-host-slash\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088441 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-system-cni-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088439 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-env-overrides\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088514 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-var-lib-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.088556 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088554 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-run-netns\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088453 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spv7r\" (UniqueName: \"kubernetes.io/projected/902ddd8c-1f3c-423f-bb33-498fcd687380-kube-api-access-spv7r\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088591 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-os-release\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088620 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088661 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t22lr\" (UniqueName: \"kubernetes.io/projected/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-kube-api-access-t22lr\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088681 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-node-log\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-netns\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a49d418-33d8-4566-aed2-dad75f7401bb-hosts-file\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088763 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-ovnkube-script-lib\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088773 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysctl-conf\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088800 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vjcn\" (UniqueName: \"kubernetes.io/projected/3fdee555-6f54-463e-aa51-aca5ba8ba135-kube-api-access-6vjcn\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088839 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-system-cni-dir\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088863 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-os-release\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088908 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088929 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47fe965a-1957-408e-8cf6-725c09aeccbc-ovnkube-config\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldh7n\" (UniqueName: \"kubernetes.io/projected/3a49d418-33d8-4566-aed2-dad75f7401bb-kube-api-access-ldh7n\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.089100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088973 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a49d418-33d8-4566-aed2-dad75f7401bb-hosts-file\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.088981 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-registration-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-slash\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089047 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/902ddd8c-1f3c-423f-bb33-498fcd687380-serviceca\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089065 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089069 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-cni-bin\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089069 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-netns\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089109 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-slash\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089124 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-cni-bin\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089130 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-var-lib-kubelet\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089157 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-registration-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089162 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06f5868c-0a31-4d2f-badf-3c0fa2a950a0-agent-certs\") pod \"konnectivity-agent-4gg28\" (UID: \"06f5868c-0a31-4d2f-badf-3c0fa2a950a0\") " pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-daemon-config\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089166 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-system-cni-dir\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089172 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089284 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.089838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-ovn\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089354 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089367 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-socket-dir-parent\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-ovn\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-etc-kubernetes\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-socket-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089440 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-etc-kubernetes\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089492 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089492 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-socket-dir-parent\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-kubernetes\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089533 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-socket-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089551 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-run\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089580 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/083f72d9-be4f-4fcb-a2a3-19fd4c746884-kubelet-config\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-cni-bin\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089618 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-cni-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.090686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089656 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-cni-bin\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-cnibin\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089721 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7600187-1b9f-4f03-bf5b-6c6d91140376-cni-binary-copy\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089747 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpzbq\" (UniqueName: \"kubernetes.io/projected/47fe965a-1957-408e-8cf6-725c09aeccbc-kube-api-access-xpzbq\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089774 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089797 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-cnibin\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089809 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-cni-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089834 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089864 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a49d418-33d8-4566-aed2-dad75f7401bb-tmp-dir\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089888 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-device-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089912 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-sys-fs\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089937 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysctl-d\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089964 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-host\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089977 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.089990 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-kubelet\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-systemd\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090034 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-sys-fs\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.091512 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cnibin\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090116 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-iptables-alerter-script\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090144 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hsqn\" (UniqueName: \"kubernetes.io/projected/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-kube-api-access-7hsqn\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090164 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cnibin\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06f5868c-0a31-4d2f-badf-3c0fa2a950a0-konnectivity-ca\") pod \"konnectivity-agent-4gg28\" (UID: \"06f5868c-0a31-4d2f-badf-3c0fa2a950a0\") " pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-run-systemd\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090217 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/083f72d9-be4f-4fcb-a2a3-19fd4c746884-dbus\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090223 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a49d418-33d8-4566-aed2-dad75f7401bb-tmp-dir\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090240 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090245 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-log-socket\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090118 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-kubelet\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090085 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-device-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090286 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-cni-netd\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090292 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7600187-1b9f-4f03-bf5b-6c6d91140376-cni-binary-copy\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090307 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-log-socket\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090316 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-hostroot\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.092287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-etc-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090372 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-host-cni-netd\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47fe965a-1957-408e-8cf6-725c09aeccbc-ovn-node-metrics-cert\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090409 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-etc-openvswitch\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090427 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-hostroot\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/902ddd8c-1f3c-423f-bb33-498fcd687380-host\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-k8s-cni-cncf-io\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090600 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-systemd-units\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090611 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/902ddd8c-1f3c-423f-bb33-498fcd687380-host\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090626 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-kubelet\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-conf-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090663 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47fe965a-1957-408e-8cf6-725c09aeccbc-systemd-units\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/bfe7c78e-0e82-4e62-8c41-0585a45658b5-kube-api-access-55tg9\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-multus-certs\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090705 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-var-lib-kubelet\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090716 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-multus-conf-dir\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090734 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090772 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-multus-certs\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.093166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090780 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090782 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090679 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7600187-1b9f-4f03-bf5b-6c6d91140376-host-run-k8s-cni-cncf-io\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090836 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfe7c78e-0e82-4e62-8c41-0585a45658b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090868 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-lib-modules\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090895 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-tuned\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.090924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxp7p\" (UniqueName: \"kubernetes.io/projected/1fe07317-22f5-4710-9349-22d9672475e1-kube-api-access-xxp7p\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:58.094013 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.091201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.095193 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.094974 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47fe965a-1957-408e-8cf6-725c09aeccbc-ovn-node-metrics-cert\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.096021 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.095690 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:06:58.096021 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.095714 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:06:58.096021 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.095729 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:06:58.096021 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.095801 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:06:58.595776891 +0000 UTC m=+3.041262001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:06:58.097875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.097727 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxwkm\" (UniqueName: \"kubernetes.io/projected/e7600187-1b9f-4f03-bf5b-6c6d91140376-kube-api-access-lxwkm\") pod \"multus-xb5hm\" (UID: \"e7600187-1b9f-4f03-bf5b-6c6d91140376\") " pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.097980 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.097875 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldh7n\" (UniqueName: \"kubernetes.io/projected/3a49d418-33d8-4566-aed2-dad75f7401bb-kube-api-access-ldh7n\") pod \"node-resolver-zgdp2\" (UID: \"3a49d418-33d8-4566-aed2-dad75f7401bb\") " pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.098162 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.098140 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t22lr\" (UniqueName: \"kubernetes.io/projected/fa000e92-7ae2-4662-aa9f-11a3c2ff42cd-kube-api-access-t22lr\") pod \"multus-additional-cni-plugins-29ggz\" (UID: \"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd\") " pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.098239 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.098140 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spv7r\" (UniqueName: \"kubernetes.io/projected/902ddd8c-1f3c-423f-bb33-498fcd687380-kube-api-access-spv7r\") pod \"node-ca-lnw96\" (UID: \"902ddd8c-1f3c-423f-bb33-498fcd687380\") " pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.099603 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.099577 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpzbq\" (UniqueName: \"kubernetes.io/projected/47fe965a-1957-408e-8cf6-725c09aeccbc-kube-api-access-xpzbq\") pod \"ovnkube-node-chzp5\" (UID: \"47fe965a-1957-408e-8cf6-725c09aeccbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.100341 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.100319 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/bfe7c78e-0e82-4e62-8c41-0585a45658b5-kube-api-access-55tg9\") pod \"aws-ebs-csi-driver-node-vdcsz\" (UID: \"bfe7c78e-0e82-4e62-8c41-0585a45658b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.147489 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.147460 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:06:58.187597 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.187569 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:06:58.191891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191861 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-kubernetes\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191895 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-run\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191922 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/083f72d9-be4f-4fcb-a2a3-19fd4c746884-kubelet-config\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysctl-d\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191961 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-kubernetes\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191963 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-run\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.191976 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-host\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192011 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-iptables-alerter-script\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.192019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192023 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-host\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192036 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hsqn\" (UniqueName: \"kubernetes.io/projected/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-kube-api-access-7hsqn\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06f5868c-0a31-4d2f-badf-3c0fa2a950a0-konnectivity-ca\") pod \"konnectivity-agent-4gg28\" (UID: \"06f5868c-0a31-4d2f-badf-3c0fa2a950a0\") " pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192091 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysctl-d\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/083f72d9-be4f-4fcb-a2a3-19fd4c746884-dbus\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192145 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/083f72d9-be4f-4fcb-a2a3-19fd4c746884-kubelet-config\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192162 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-lib-modules\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.192455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192252 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/083f72d9-be4f-4fcb-a2a3-19fd4c746884-dbus\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-tuned\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192567 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxp7p\" (UniqueName: \"kubernetes.io/projected/1fe07317-22f5-4710-9349-22d9672475e1-kube-api-access-xxp7p\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192573 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-iptables-alerter-script\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192596 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-systemd\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fdee555-6f54-463e-aa51-aca5ba8ba135-tmp\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192630 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06f5868c-0a31-4d2f-badf-3c0fa2a950a0-konnectivity-ca\") pod \"konnectivity-agent-4gg28\" (UID: \"06f5868c-0a31-4d2f-badf-3c0fa2a950a0\") " pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192640 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-sys\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysconfig\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192674 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-host-slash\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192723 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysctl-conf\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vjcn\" (UniqueName: \"kubernetes.io/projected/3fdee555-6f54-463e-aa51-aca5ba8ba135-kube-api-access-6vjcn\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192758 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-var-lib-kubelet\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192778 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06f5868c-0a31-4d2f-badf-3c0fa2a950a0-agent-certs\") pod \"konnectivity-agent-4gg28\" (UID: \"06f5868c-0a31-4d2f-badf-3c0fa2a950a0\") " pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192795 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192751 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-lib-modules\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.192886 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:06:58.193379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192911 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-sys\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.192947 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:06:58.692929551 +0000 UTC m=+3.138414680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192970 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysconfig\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.192990 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-host-slash\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.193025 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-var-lib-kubelet\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.193169 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-sysctl-conf\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.193582 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-systemd\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.193870 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:06:58.194306 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.193922 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:06:58.69390317 +0000 UTC m=+3.139388283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:06:58.194838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.194802 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lnw96" Jul 12 08:06:58.195760 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.195735 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06f5868c-0a31-4d2f-badf-3c0fa2a950a0-agent-certs\") pod \"konnectivity-agent-4gg28\" (UID: \"06f5868c-0a31-4d2f-badf-3c0fa2a950a0\") " pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.195988 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.195965 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3fdee555-6f54-463e-aa51-aca5ba8ba135-etc-tuned\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.196138 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.196107 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fdee555-6f54-463e-aa51-aca5ba8ba135-tmp\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.202004 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.201969 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-29ggz" Jul 12 08:06:58.202733 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.202699 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vjcn\" (UniqueName: \"kubernetes.io/projected/3fdee555-6f54-463e-aa51-aca5ba8ba135-kube-api-access-6vjcn\") pod \"tuned-z5tzl\" (UID: \"3fdee555-6f54-463e-aa51-aca5ba8ba135\") " pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.203348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.203326 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hsqn\" (UniqueName: \"kubernetes.io/projected/86ae5582-d6b4-4de3-8bdb-0e54adc2561a-kube-api-access-7hsqn\") pod \"iptables-alerter-d268w\" (UID: \"86ae5582-d6b4-4de3-8bdb-0e54adc2561a\") " pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.204323 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.204296 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47fe965a_1957_408e_8cf6_725c09aeccbc.slice/crio-d2ffe6f305e76191e5efa69cc37089f2bc8c45fef1fbdb35bc37894883b60a44 WatchSource:0}: Error finding container d2ffe6f305e76191e5efa69cc37089f2bc8c45fef1fbdb35bc37894883b60a44: Status 404 returned error can't find the container with id d2ffe6f305e76191e5efa69cc37089f2bc8c45fef1fbdb35bc37894883b60a44 Jul 12 08:06:58.205358 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.205157 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxp7p\" (UniqueName: \"kubernetes.io/projected/1fe07317-22f5-4710-9349-22d9672475e1-kube-api-access-xxp7p\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:58.210785 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.207519 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xb5hm" Jul 12 08:06:58.210785 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.209988 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902ddd8c_1f3c_423f_bb33_498fcd687380.slice/crio-63e21875b2b518cdcbe550cb355c5251692f360c2b219eed7679e42c6ef76392 WatchSource:0}: Error finding container 63e21875b2b518cdcbe550cb355c5251692f360c2b219eed7679e42c6ef76392: Status 404 returned error can't find the container with id 63e21875b2b518cdcbe550cb355c5251692f360c2b219eed7679e42c6ef76392 Jul 12 08:06:58.214703 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.214681 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d268w" Jul 12 08:06:58.219159 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.218570 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa000e92_7ae2_4662_aa9f_11a3c2ff42cd.slice/crio-5659855f9180aa4b7d3c4a560c3456010997c84b9608c831af01f0cba538b799 WatchSource:0}: Error finding container 5659855f9180aa4b7d3c4a560c3456010997c84b9608c831af01f0cba538b799: Status 404 returned error can't find the container with id 5659855f9180aa4b7d3c4a560c3456010997c84b9608c831af01f0cba538b799 Jul 12 08:06:58.219575 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.219555 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:06:58.223441 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.223388 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7600187_1b9f_4f03_bf5b_6c6d91140376.slice/crio-122f1d2cc4b823f389698adcf10096acc71e890849d40fa52cac36e728fb0275 WatchSource:0}: Error finding container 122f1d2cc4b823f389698adcf10096acc71e890849d40fa52cac36e728fb0275: Status 404 returned error can't find the container with id 122f1d2cc4b823f389698adcf10096acc71e890849d40fa52cac36e728fb0275 Jul 12 08:06:58.226131 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.226107 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zgdp2" Jul 12 08:06:58.231948 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.231924 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" Jul 12 08:06:58.236808 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.236785 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" Jul 12 08:06:58.239373 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.239314 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ae5582_d6b4_4de3_8bdb_0e54adc2561a.slice/crio-6d5bc9b7abebabeba9652c51eed362af2e0ce4909e8d77af7eb7a2f1a58f027c WatchSource:0}: Error finding container 6d5bc9b7abebabeba9652c51eed362af2e0ce4909e8d77af7eb7a2f1a58f027c: Status 404 returned error can't find the container with id 6d5bc9b7abebabeba9652c51eed362af2e0ce4909e8d77af7eb7a2f1a58f027c Jul 12 08:06:58.249831 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.249774 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f5868c_0a31_4d2f_badf_3c0fa2a950a0.slice/crio-0ab43dab16ddfa5a066b29ebb97fa11b0d3182025d1aae1530d9a3bcd458d86b WatchSource:0}: Error finding container 0ab43dab16ddfa5a066b29ebb97fa11b0d3182025d1aae1530d9a3bcd458d86b: Status 404 returned error can't find the container with id 0ab43dab16ddfa5a066b29ebb97fa11b0d3182025d1aae1530d9a3bcd458d86b Jul 12 08:06:58.251598 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.251568 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe7c78e_0e82_4e62_8c41_0585a45658b5.slice/crio-c1986df625415203a41de564d9235a78ae95e56c42099a33d520232894cbe4cc WatchSource:0}: Error finding container c1986df625415203a41de564d9235a78ae95e56c42099a33d520232894cbe4cc: Status 404 returned error can't find the container with id c1986df625415203a41de564d9235a78ae95e56c42099a33d520232894cbe4cc Jul 12 08:06:58.253156 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.252710 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a49d418_33d8_4566_aed2_dad75f7401bb.slice/crio-56380b393159e86384341f1f2136d62f7a66da6b297d554090b352e0be26e09a WatchSource:0}: Error finding container 56380b393159e86384341f1f2136d62f7a66da6b297d554090b352e0be26e09a: Status 404 returned error can't find the container with id 56380b393159e86384341f1f2136d62f7a66da6b297d554090b352e0be26e09a Jul 12 08:06:58.255547 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:06:58.255457 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdee555_6f54_463e_aa51_aca5ba8ba135.slice/crio-38af9269876e7fe123d9203b31f9af497d9212a895ce4ca08f6a0b53d060f65c WatchSource:0}: Error finding container 38af9269876e7fe123d9203b31f9af497d9212a895ce4ca08f6a0b53d060f65c: Status 404 returned error can't find the container with id 38af9269876e7fe123d9203b31f9af497d9212a895ce4ca08f6a0b53d060f65c Jul 12 08:06:58.388886 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.388774 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" event={"ID":"bfe7c78e-0e82-4e62-8c41-0585a45658b5","Type":"ContainerStarted","Data":"c1986df625415203a41de564d9235a78ae95e56c42099a33d520232894cbe4cc"} Jul 12 08:06:58.390094 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.390058 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zgdp2" event={"ID":"3a49d418-33d8-4566-aed2-dad75f7401bb","Type":"ContainerStarted","Data":"56380b393159e86384341f1f2136d62f7a66da6b297d554090b352e0be26e09a"} Jul 12 08:06:58.392714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.392313 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d268w" event={"ID":"86ae5582-d6b4-4de3-8bdb-0e54adc2561a","Type":"ContainerStarted","Data":"6d5bc9b7abebabeba9652c51eed362af2e0ce4909e8d77af7eb7a2f1a58f027c"} Jul 12 08:06:58.395463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.395430 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xb5hm" event={"ID":"e7600187-1b9f-4f03-bf5b-6c6d91140376","Type":"ContainerStarted","Data":"122f1d2cc4b823f389698adcf10096acc71e890849d40fa52cac36e728fb0275"} Jul 12 08:06:58.397137 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.397111 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"d2ffe6f305e76191e5efa69cc37089f2bc8c45fef1fbdb35bc37894883b60a44"} Jul 12 08:06:58.398414 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.398379 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" event={"ID":"adf1dd458608a3497f114fbce5c49409","Type":"ContainerStarted","Data":"8163f1a84026091650647889b116a57623eee885391875c97779d50320fb7e8a"} Jul 12 08:06:58.400094 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.400010 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" event={"ID":"477ee7992b12ed0719fbc7684ecfecc6","Type":"ContainerStarted","Data":"ef296ba05210d35d81f6f3cf18a5fe7ca237aa0908f649b82cf6e6edba1f6fc3"} Jul 12 08:06:58.401312 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.401289 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" event={"ID":"3fdee555-6f54-463e-aa51-aca5ba8ba135","Type":"ContainerStarted","Data":"38af9269876e7fe123d9203b31f9af497d9212a895ce4ca08f6a0b53d060f65c"} Jul 12 08:06:58.402490 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.402460 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4gg28" event={"ID":"06f5868c-0a31-4d2f-badf-3c0fa2a950a0","Type":"ContainerStarted","Data":"0ab43dab16ddfa5a066b29ebb97fa11b0d3182025d1aae1530d9a3bcd458d86b"} Jul 12 08:06:58.403801 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.403748 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerStarted","Data":"5659855f9180aa4b7d3c4a560c3456010997c84b9608c831af01f0cba538b799"} Jul 12 08:06:58.405032 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.405006 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lnw96" event={"ID":"902ddd8c-1f3c-423f-bb33-498fcd687380","Type":"ContainerStarted","Data":"63e21875b2b518cdcbe550cb355c5251692f360c2b219eed7679e42c6ef76392"} Jul 12 08:06:58.596396 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.596342 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:06:58.596618 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.596594 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:06:58.596681 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.596626 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:06:58.596681 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.596642 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:06:58.596800 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.596709 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:06:59.596692478 +0000 UTC m=+4.042177587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:06:58.697153 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.697064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:58.697153 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.697123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:58.697358 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.697242 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:06:58.697358 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.697244 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:06:58.697358 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.697309 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:06:59.697291152 +0000 UTC m=+4.142776263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:06:58.697358 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:58.697328 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:06:59.697318222 +0000 UTC m=+4.142803331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:06:58.934379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.934313 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 08:01:58 +0000 UTC" deadline="2028-03-28 17:16:01.377378436 +0000 UTC" Jul 12 08:06:58.934379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:58.934372 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15009h9m2.443010827s" Jul 12 08:06:59.382453 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.382420 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:06:59.383200 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.382543 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:06:59.383200 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.382593 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:59.383200 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.382596 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:59.383200 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.382705 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:06:59.383200 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.382827 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:06:59.606483 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.606447 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:06:59.606653 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.606630 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:06:59.606719 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.606655 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:06:59.606719 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.606670 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:06:59.606853 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.606732 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:01.606713168 +0000 UTC m=+6.052198284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:06:59.708220 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.708094 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:06:59.708220 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.708158 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:06:59.708415 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.708292 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:06:59.708415 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.708352 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:01.708335304 +0000 UTC m=+6.153820427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:06:59.708856 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.708836 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:06:59.708934 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:06:59.708908 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:01.708881483 +0000 UTC m=+6.154366592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:06:59.934982 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.934932 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 08:01:58 +0000 UTC" deadline="2028-01-25 05:42:11.869481897 +0000 UTC" Jul 12 08:06:59.934982 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:06:59.934972 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13485h35m11.934516369s" Jul 12 08:07:01.381729 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:01.381698 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:01.382239 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.381843 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:01.382239 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:01.382232 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:01.382543 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.382358 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:01.382543 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:01.382411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:01.382543 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.382474 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:01.626098 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:01.626059 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:01.626340 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.626293 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:07:01.626340 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.626323 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:07:01.626340 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.626336 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:01.626512 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.626390 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:05.626372864 +0000 UTC m=+10.071857977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:01.727055 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:01.726934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:01.727055 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:01.727021 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:01.727274 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.727151 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:01.727274 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.727204 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:05.727187653 +0000 UTC m=+10.172672767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:01.727274 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.727235 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:01.727431 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:01.727301 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:05.727283881 +0000 UTC m=+10.172769018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:03.382202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:03.382119 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:03.382202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:03.382158 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:03.382701 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:03.382261 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:03.382701 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:03.382624 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:03.382847 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:03.382733 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:03.382847 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:03.382837 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:05.381534 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:05.381472 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:05.381534 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:05.381505 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:05.382074 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.381599 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:05.382074 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:05.381952 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:05.382074 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.382056 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:05.382227 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.382133 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:05.663598 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:05.663516 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:05.663781 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.663678 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:07:05.663781 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.663697 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:07:05.663781 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.663710 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:05.663781 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.663781 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:13.663763765 +0000 UTC m=+18.109248891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:05.765099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:05.764568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:05.765099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:05.764629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:05.765099 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.764772 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:05.765099 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.764831 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:13.764813668 +0000 UTC m=+18.210298788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:05.765099 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.764772 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:05.765099 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:05.764924 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:13.764904141 +0000 UTC m=+18.210389262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:07.381517 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:07.381482 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:07.381979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:07.381528 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:07.381979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:07.381553 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:07.381979 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:07.381668 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:07.381979 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:07.381850 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:07.381979 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:07.381946 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:09.381784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:09.381759 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:09.382155 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:09.381760 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:09.382155 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:09.381891 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:09.382155 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:09.381760 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:09.382155 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:09.381971 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:09.382155 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:09.382040 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:11.381930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:11.381891 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:11.381930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:11.381916 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:11.382408 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:11.381939 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:11.382408 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:11.382015 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:11.382408 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:11.382084 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:11.382408 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:11.382185 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:13.382130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:13.382096 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:13.382572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:13.382147 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:13.382572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:13.382207 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:13.382572 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.382224 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:13.382572 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.382276 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:13.382572 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.382361 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:13.723845 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:13.723753 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:13.724013 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.723908 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:07:13.724013 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.723941 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:07:13.724013 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.723956 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:13.724156 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.724020 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:29.724000354 +0000 UTC m=+34.169485487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:13.825098 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:13.825065 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:13.825292 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:13.825136 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:13.825292 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.825221 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:13.825292 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.825224 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:13.825292 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.825280 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:29.825265133 +0000 UTC m=+34.270750242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:13.825454 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:13.825297 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:07:29.8252903 +0000 UTC m=+34.270775408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:15.381820 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:15.381603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:15.382286 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:15.381603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:15.382286 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:15.381917 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:15.382286 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:15.381603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:15.382286 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:15.381985 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:15.382286 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:15.382069 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:17.382272 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:17.382244 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:17.382272 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:17.382267 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:17.382643 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:17.382245 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:17.382643 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:17.382340 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:17.382643 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:17.382418 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:17.382643 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:17.382470 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:18.444705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.444375 2558 generic.go:356] "Generic (PLEG): container finished" podID="fa000e92-7ae2-4662-aa9f-11a3c2ff42cd" containerID="7106e5d475ab8c2b2dd713aa02dc3b884201b30d0428dc674eb2e98e2d344f27" exitCode=0 Jul 12 08:07:18.445444 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.444453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerDied","Data":"7106e5d475ab8c2b2dd713aa02dc3b884201b30d0428dc674eb2e98e2d344f27"} Jul 12 08:07:18.446333 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.446308 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lnw96" event={"ID":"902ddd8c-1f3c-423f-bb33-498fcd687380","Type":"ContainerStarted","Data":"ddddf7b834e405a2d9624e10a4a943bbd1fa68208160432385628c8af617bb52"} Jul 12 08:07:18.447946 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.447918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" event={"ID":"bfe7c78e-0e82-4e62-8c41-0585a45658b5","Type":"ContainerStarted","Data":"cc506e3af7cfd041b9e200c5412190d7e45ae1ef516c986102d479d4866626f7"} Jul 12 08:07:18.449409 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.449360 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zgdp2" event={"ID":"3a49d418-33d8-4566-aed2-dad75f7401bb","Type":"ContainerStarted","Data":"ac63504e7f895bc1a0745717531a9e1e15d37e2f4f46f9506b52a9cc3448993c"} Jul 12 08:07:18.450996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.450973 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xb5hm" event={"ID":"e7600187-1b9f-4f03-bf5b-6c6d91140376","Type":"ContainerStarted","Data":"3eceedc1ab260ab6692b1e35db4e633be861e3cfe35bf8d960eda54bac7b833a"} Jul 12 08:07:18.454050 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454030 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:07:18.454345 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454324 2558 generic.go:356] "Generic (PLEG): container finished" podID="47fe965a-1957-408e-8cf6-725c09aeccbc" containerID="5f1b9ced8cedce9444cfdc790bc8826f7da4494adb1d26e0885bd49c7f1f85c9" exitCode=1 Jul 12 08:07:18.454448 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454389 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"f55b0ae80b1d517fae7feb9baa3bb4e5b0c3eec7885e445b1688bd65371ca5e7"} Jul 12 08:07:18.454517 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454463 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"dff3cc2f00cf48a97ef43973fed0804271038e8bdfe298066e85bd91a67e9a41"} Jul 12 08:07:18.454572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454517 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"6e60f5e6e9812270199084dfb4191d14b89bd97b4cdafa4098972da404513e3d"} Jul 12 08:07:18.454572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454566 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"02de367377370d5e3a6ccd113d4338b38f684d2b32f9ed1154345fa0a09e876e"} Jul 12 08:07:18.454673 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454613 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerDied","Data":"5f1b9ced8cedce9444cfdc790bc8826f7da4494adb1d26e0885bd49c7f1f85c9"} Jul 12 08:07:18.454673 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.454664 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"3e4537eb2b2d7afbe6c519f5a3847c575b499b409b30c32ebb3886958ac2c29b"} Jul 12 08:07:18.455899 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.455876 2558 generic.go:356] "Generic (PLEG): container finished" podID="adf1dd458608a3497f114fbce5c49409" containerID="56bcebc960bec41737e77a3ae497aaf841fe16af35193126d5b71e755504b499" exitCode=0 Jul 12 08:07:18.455981 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.455954 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" event={"ID":"adf1dd458608a3497f114fbce5c49409","Type":"ContainerDied","Data":"56bcebc960bec41737e77a3ae497aaf841fe16af35193126d5b71e755504b499"} Jul 12 08:07:18.457688 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.457637 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" event={"ID":"477ee7992b12ed0719fbc7684ecfecc6","Type":"ContainerStarted","Data":"a6ea3cef61888e74f2d57ab93785ec56600780f875ecb64207d5b4f097a628df"} Jul 12 08:07:18.459559 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.459453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" event={"ID":"3fdee555-6f54-463e-aa51-aca5ba8ba135","Type":"ContainerStarted","Data":"e335b2a7f30147c180ddbb0b5f012ba41cdc495333540c193bf992ab4ddfffde"} Jul 12 08:07:18.461142 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.461122 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4gg28" event={"ID":"06f5868c-0a31-4d2f-badf-3c0fa2a950a0","Type":"ContainerStarted","Data":"eefa5446c6378939c006ed1236ddb3811f4577639ad6081eb434a19e998f08d4"} Jul 12 08:07:18.477014 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.476965 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-48.ec2.internal" podStartSLOduration=21.476950315 podStartE2EDuration="21.476950315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:18.476903196 +0000 UTC m=+22.922388327" watchObservedRunningTime="2026-07-12 08:07:18.476950315 +0000 UTC m=+22.922435445" Jul 12 08:07:18.492591 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.492544 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lnw96" podStartSLOduration=7.169125198 podStartE2EDuration="22.492523525s" totalImagesPullingTime="15.323398327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:18.492211728 +0000 UTC m=+22.937696870" watchObservedRunningTime="2026-07-12 08:07:18.492523525 +0000 UTC m=+22.938008648" Jul 12 08:07:18.507114 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.507063 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xb5hm" podStartSLOduration=3.021344287 podStartE2EDuration="22.507044221s" totalImagesPullingTime="19.485699934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:18.5069216 +0000 UTC m=+22.952406732" watchObservedRunningTime="2026-07-12 08:07:18.507044221 +0000 UTC m=+22.952529352" Jul 12 08:07:18.536438 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.536380 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4gg28" podStartSLOduration=3.130156939 podStartE2EDuration="22.536360134s" totalImagesPullingTime="19.406203195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:18.522599669 +0000 UTC m=+22.968084799" watchObservedRunningTime="2026-07-12 08:07:18.536360134 +0000 UTC m=+22.981845261" Jul 12 08:07:18.536875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.536840 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zgdp2" podStartSLOduration=3.161125707 podStartE2EDuration="22.536830158s" totalImagesPullingTime="19.375704451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:18.535971504 +0000 UTC m=+22.981456660" watchObservedRunningTime="2026-07-12 08:07:18.536830158 +0000 UTC m=+22.982315290" Jul 12 08:07:18.551378 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:18.551336 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z5tzl" podStartSLOduration=3.148707409 podStartE2EDuration="22.551318779s" totalImagesPullingTime="19.40261137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:18.550717116 +0000 UTC m=+22.996202251" watchObservedRunningTime="2026-07-12 08:07:18.551318779 +0000 UTC m=+22.996803911" Jul 12 08:07:19.361880 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.361849 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 08:07:19.381891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.381829 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:19.381891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.381856 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:19.381891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.381856 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:19.382092 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:19.381933 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:19.382092 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:19.382020 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:19.382189 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:19.382147 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:19.465988 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.465952 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" event={"ID":"adf1dd458608a3497f114fbce5c49409","Type":"ContainerStarted","Data":"b870e7eb7d9176d9a3ed9857dfbc94ecd661f6fc99002ddbd5904aad30b99ec8"} Jul 12 08:07:19.468131 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.468103 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" event={"ID":"bfe7c78e-0e82-4e62-8c41-0585a45658b5","Type":"ContainerStarted","Data":"a740b801c7ef6b1cb12a299f1cc04d2883471df11ba29be28cc990e7411f3de0"} Jul 12 08:07:19.469600 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.469569 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d268w" event={"ID":"86ae5582-d6b4-4de3-8bdb-0e54adc2561a","Type":"ContainerStarted","Data":"4c1887d315a1489e0105b9f4501fa380ce67e2920526ca99572dae78af4920ba"} Jul 12 08:07:19.479471 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.479422 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-48.ec2.internal" podStartSLOduration=22.479406059 podStartE2EDuration="22.479406059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:19.478963675 +0000 UTC m=+23.924448806" watchObservedRunningTime="2026-07-12 08:07:19.479406059 +0000 UTC m=+23.924891208" Jul 12 08:07:19.491952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.491901 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d268w" podStartSLOduration=8.198297763 podStartE2EDuration="23.491884771s" totalImagesPullingTime="15.293587008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:19.491111402 +0000 UTC m=+23.936596532" watchObservedRunningTime="2026-07-12 08:07:19.491884771 +0000 UTC m=+23.937369902" Jul 12 08:07:19.684667 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.684592 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:19.767594 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:19.767547 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:20.155352 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:20.155311 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:20.342991 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:20.342883 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T08:07:19.361876044Z","UUID":"532de573-2776-4ad8-937a-7db2a97f8e76","Handler":null,"Name":"","Endpoint":""} Jul 12 08:07:20.344540 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:20.344517 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 08:07:20.344662 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:20.344561 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 08:07:21.382108 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.382074 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:21.382609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.382116 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:21.382609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.382116 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:21.382609 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:21.382197 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:21.382609 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:21.382305 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:21.382609 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:21.382388 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:21.477316 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.477283 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:07:21.477755 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.477711 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"752549bbce8e33dfe98b8b10ab42e8869127377cbd9404b2087ae557aa107d16"} Jul 12 08:07:21.480156 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.480058 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" event={"ID":"bfe7c78e-0e82-4e62-8c41-0585a45658b5","Type":"ContainerStarted","Data":"46899b51db76f6e2f0bee28ade0d554ebf6d9bd95bbb07f817066db6f6e59384"} Jul 12 08:07:21.496378 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.496340 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcsz" podStartSLOduration=3.8753606659999997 podStartE2EDuration="25.496328071s" totalImagesPullingTime="21.620967405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:21.495692226 +0000 UTC m=+25.941177358" watchObservedRunningTime="2026-07-12 08:07:21.496328071 +0000 UTC m=+25.941813200" Jul 12 08:07:21.887046 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.886839 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:07:21.887484 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:21.887465 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:07:22.482292 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:22.482262 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:07:22.483205 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:22.483184 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4gg28" Jul 12 08:07:22.534820 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:22.534791 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:23.011793 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.011763 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:23.111598 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.111572 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:23.381844 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.381652 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:23.382012 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.381707 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:23.382012 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:23.381912 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:23.382012 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.381711 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:23.382012 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:23.381989 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:23.382147 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:23.382058 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:23.486924 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.486897 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:07:23.487715 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.487276 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"e1edb6c94275f316201fc202c86ab79f55f447f41136365fa236e4f86a78abec"} Jul 12 08:07:23.487715 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.487494 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:07:23.487715 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.487660 2558 scope.go:117] "RemoveContainer" containerID="5f1b9ced8cedce9444cfdc790bc8826f7da4494adb1d26e0885bd49c7f1f85c9" Jul 12 08:07:23.489178 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.489151 2558 generic.go:356] "Generic (PLEG): container finished" podID="fa000e92-7ae2-4662-aa9f-11a3c2ff42cd" containerID="0957dc0424724f9e18334c83d776ae98e83c5533415afe16bc74cfd39b7d149a" exitCode=0 Jul 12 08:07:23.489280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.489192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerDied","Data":"0957dc0424724f9e18334c83d776ae98e83c5533415afe16bc74cfd39b7d149a"} Jul 12 08:07:23.503424 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:23.503405 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:07:24.257503 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.257422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:24.312498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.312461 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:24.388009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.387962 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:24.489664 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.489620 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:24.494861 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.494841 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:07:24.495267 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.495238 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" event={"ID":"47fe965a-1957-408e-8cf6-725c09aeccbc","Type":"ContainerStarted","Data":"6eab8c49d22925034838fdc3b7f56b488c84edb4ceb5aedac69e1a569839ab1c"} Jul 12 08:07:24.495373 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.495336 2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 08:07:24.495568 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.495550 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:07:24.511433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.511372 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:07:24.523051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.523011 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" podStartSLOduration=8.998041644 podStartE2EDuration="28.522995407s" totalImagesPullingTime="19.524953763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:24.521633304 +0000 UTC m=+28.967118445" watchObservedRunningTime="2026-07-12 08:07:24.522995407 +0000 UTC m=+28.968480540" Jul 12 08:07:24.812047 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.812017 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8gf2t"] Jul 12 08:07:24.812226 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.812175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:24.812310 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:24.812283 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:24.815279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.815253 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xsdxv"] Jul 12 08:07:24.815435 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.815355 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:24.815435 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:24.815428 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:24.815857 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.815833 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gjm7q"] Jul 12 08:07:24.815967 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.815936 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:24.816043 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:24.816025 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:24.914594 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:24.914559 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:07:25.112870 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:25.112721 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:25.499867 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:25.499786 2558 generic.go:356] "Generic (PLEG): container finished" podID="fa000e92-7ae2-4662-aa9f-11a3c2ff42cd" containerID="d3c0b51c120f814371cafcf97e896628c922c1b23ca217445bd59b6162be612d" exitCode=0 Jul 12 08:07:25.500237 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:25.499886 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerDied","Data":"d3c0b51c120f814371cafcf97e896628c922c1b23ca217445bd59b6162be612d"} Jul 12 08:07:26.382707 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:26.382678 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:26.382901 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:26.382802 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:26.382901 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:26.382874 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:26.383020 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:26.382963 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:26.383020 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:26.383000 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:26.383116 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:26.383059 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:27.506997 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:27.506787 2558 generic.go:356] "Generic (PLEG): container finished" podID="fa000e92-7ae2-4662-aa9f-11a3c2ff42cd" containerID="dc9b5a658e39030046555209368561d58c20b4bea5c0814f5440c846d3469504" exitCode=0 Jul 12 08:07:27.506997 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:27.506871 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerDied","Data":"dc9b5a658e39030046555209368561d58c20b4bea5c0814f5440c846d3469504"} Jul 12 08:07:28.385258 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:28.385233 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:28.385258 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:28.385244 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:28.385501 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:28.385235 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:28.385501 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:28.385344 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xsdxv" podUID="706842c4-d3bb-42f3-8168-db36876a85c9" Jul 12 08:07:28.385501 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:28.385419 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjm7q" podUID="1fe07317-22f5-4710-9349-22d9672475e1" Jul 12 08:07:28.385664 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:28.385518 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gf2t" podUID="083f72d9-be4f-4fcb-a2a3-19fd4c746884" Jul 12 08:07:29.746993 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:29.746956 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:29.747579 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.747081 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:07:29.747579 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.747095 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:07:29.747579 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.747103 2558 projected.go:196] Error preparing data for projected volume kube-api-access-29l9x for pod openshift-network-diagnostics/network-check-target-xsdxv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:29.747579 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.747150 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x podName:706842c4-d3bb-42f3-8168-db36876a85c9 nodeName:}" failed. No retries permitted until 2026-07-12 08:08:01.747136681 +0000 UTC m=+66.192621788 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-29l9x" (UniqueName: "kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x") pod "network-check-target-xsdxv" (UID: "706842c4-d3bb-42f3-8168-db36876a85c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:07:29.848196 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:29.848165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:29.848472 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:29.848260 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:29.848472 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.848323 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:29.848472 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.848353 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:29.848472 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.848386 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret podName:083f72d9-be4f-4fcb-a2a3-19fd4c746884 nodeName:}" failed. No retries permitted until 2026-07-12 08:08:01.84836982 +0000 UTC m=+66.293854939 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret") pod "global-pull-secret-syncer-8gf2t" (UID: "083f72d9-be4f-4fcb-a2a3-19fd4c746884") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:07:29.848472 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:29.848402 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs podName:1fe07317-22f5-4710-9349-22d9672475e1 nodeName:}" failed. No retries permitted until 2026-07-12 08:08:01.848394416 +0000 UTC m=+66.293879524 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs") pod "network-metrics-daemon-gjm7q" (UID: "1fe07317-22f5-4710-9349-22d9672475e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:07:30.276510 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.276455 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:30.332365 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.332328 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:30.385087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.385057 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-48.ec2.internal" event="NodeReady" Jul 12 08:07:30.385234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.385165 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:07:30.385234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.385212 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 08:07:30.385639 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.385464 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:07:30.385775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.385646 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:07:30.387389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.387361 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rkfrj\"" Jul 12 08:07:30.387568 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.387544 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 08:07:30.387659 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.387612 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 08:07:30.388163 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.388129 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xxbfq\"" Jul 12 08:07:30.388262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.388185 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 08:07:30.388262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.388189 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 08:07:30.432911 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.432886 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-grhbr"] Jul 12 08:07:30.437330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.437307 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.437698 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.437675 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l8z9l"] Jul 12 08:07:30.438877 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.438825 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:30.439379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.439361 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 08:07:30.439472 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.439454 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 08:07:30.439549 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.439527 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lhqtd\"" Jul 12 08:07:30.440791 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.440775 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.442756 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.442721 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 08:07:30.442885 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.442870 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 08:07:30.443014 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.442999 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 08:07:30.443216 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.443200 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mn9v7\"" Jul 12 08:07:30.446610 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.446588 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-grhbr"] Jul 12 08:07:30.449783 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.449763 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l8z9l"] Jul 12 08:07:30.554022 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.553984 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrsbv\" (UniqueName: \"kubernetes.io/projected/09bc5f43-1026-4cdc-8a41-eae824bd5d7c-kube-api-access-vrsbv\") pod \"ingress-canary-l8z9l\" (UID: \"09bc5f43-1026-4cdc-8a41-eae824bd5d7c\") " pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.554022 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.554025 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.554261 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.554096 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-tmp-dir\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.554261 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.554142 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49trs\" (UniqueName: \"kubernetes.io/projected/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-kube-api-access-49trs\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.554261 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.554202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09bc5f43-1026-4cdc-8a41-eae824bd5d7c-cert\") pod \"ingress-canary-l8z9l\" (UID: \"09bc5f43-1026-4cdc-8a41-eae824bd5d7c\") " pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.554407 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.554297 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-config-volume\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.654799 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.654766 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrsbv\" (UniqueName: \"kubernetes.io/projected/09bc5f43-1026-4cdc-8a41-eae824bd5d7c-kube-api-access-vrsbv\") pod \"ingress-canary-l8z9l\" (UID: \"09bc5f43-1026-4cdc-8a41-eae824bd5d7c\") " pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.654969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.654808 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.654969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.654840 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-tmp-dir\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.654969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.654856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49trs\" (UniqueName: \"kubernetes.io/projected/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-kube-api-access-49trs\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.654969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.654883 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09bc5f43-1026-4cdc-8a41-eae824bd5d7c-cert\") pod \"ingress-canary-l8z9l\" (UID: \"09bc5f43-1026-4cdc-8a41-eae824bd5d7c\") " pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.654969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.654934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-config-volume\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.655228 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:30.655065 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:07:30.655228 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:30.655126 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls podName:8148b2db-3b6c-47ed-ba45-fcd50dd4d57d nodeName:}" failed. No retries permitted until 2026-07-12 08:07:31.155106609 +0000 UTC m=+35.600591717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls") pod "dns-default-grhbr" (UID: "8148b2db-3b6c-47ed-ba45-fcd50dd4d57d") : secret "dns-default-metrics-tls" not found Jul 12 08:07:30.655326 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.655240 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-tmp-dir\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.655541 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.655526 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-config-volume\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.659729 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.659709 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09bc5f43-1026-4cdc-8a41-eae824bd5d7c-cert\") pod \"ingress-canary-l8z9l\" (UID: \"09bc5f43-1026-4cdc-8a41-eae824bd5d7c\") " pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.663701 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.663682 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49trs\" (UniqueName: \"kubernetes.io/projected/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-kube-api-access-49trs\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:30.663968 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.663951 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrsbv\" (UniqueName: \"kubernetes.io/projected/09bc5f43-1026-4cdc-8a41-eae824bd5d7c-kube-api-access-vrsbv\") pod \"ingress-canary-l8z9l\" (UID: \"09bc5f43-1026-4cdc-8a41-eae824bd5d7c\") " pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:30.756422 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:30.756345 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l8z9l" Jul 12 08:07:31.056829 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.056729 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.116362 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.116302 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.158332 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.158297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:31.158508 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.158455 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:07:31.158561 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.158523 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls podName:8148b2db-3b6c-47ed-ba45-fcd50dd4d57d nodeName:}" failed. No retries permitted until 2026-07-12 08:07:32.158504987 +0000 UTC m=+36.603990096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls") pod "dns-default-grhbr" (UID: "8148b2db-3b6c-47ed-ba45-fcd50dd4d57d") : secret "dns-default-metrics-tls" not found Jul 12 08:07:31.242533 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.242496 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g25qv"] Jul 12 08:07:31.245848 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.245823 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.251218 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.251194 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 08:07:31.251953 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.251934 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 08:07:31.252608 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.252579 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmv4g\"" Jul 12 08:07:31.254087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.253536 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 08:07:31.254087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.253732 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 08:07:31.263566 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.263535 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g25qv"] Jul 12 08:07:31.293210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.293177 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.344257 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.344220 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.360485 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.360450 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.360642 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.360490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.360642 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.360519 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5zgl\" (UniqueName: \"kubernetes.io/projected/f6138865-5794-481a-a099-18afed93496b-kube-api-access-b5zgl\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.360642 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.360625 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f6138865-5794-481a-a099-18afed93496b-crio-socket\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.360810 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.360670 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6138865-5794-481a-a099-18afed93496b-data-volume\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462000 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.461968 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6138865-5794-481a-a099-18afed93496b-data-volume\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462164 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.462040 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462164 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.462062 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462164 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.462081 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5zgl\" (UniqueName: \"kubernetes.io/projected/f6138865-5794-481a-a099-18afed93496b-kube-api-access-b5zgl\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462164 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.462115 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f6138865-5794-481a-a099-18afed93496b-crio-socket\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462366 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.462268 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:07:31.462366 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.462286 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:07:31.462366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.462300 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f6138865-5794-481a-a099-18afed93496b-crio-socket\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.462366 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.462345 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:31.962324817 +0000 UTC m=+36.407809930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : secret "insights-runtime-extractor-tls" not found Jul 12 08:07:31.462366 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.462366 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:31.962356639 +0000 UTC m=+36.407841754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : configmap "kube-rbac-proxy" not found Jul 12 08:07:31.462366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.462361 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6138865-5794-481a-a099-18afed93496b-data-volume\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.469660 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.469631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5zgl\" (UniqueName: \"kubernetes.io/projected/f6138865-5794-481a-a099-18afed93496b-kube-api-access-b5zgl\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.524178 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.523998 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.580545 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.580520 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l8z9l"] Jul 12 08:07:31.586452 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.586427 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.588809 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:31.588784 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bc5f43_1026_4cdc_8a41_eae824bd5d7c.slice/crio-6fb0af3f43895eba7f6f60fad93e5673636b21d44e21bc0c6ebdd576fdfff2a7 WatchSource:0}: Error finding container 6fb0af3f43895eba7f6f60fad93e5673636b21d44e21bc0c6ebdd576fdfff2a7: Status 404 returned error can't find the container with id 6fb0af3f43895eba7f6f60fad93e5673636b21d44e21bc0c6ebdd576fdfff2a7 Jul 12 08:07:31.649072 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.649034 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.750467 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.750441 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.849900 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.849833 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:31.966271 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.966235 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.966439 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:31.966285 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:31.966439 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.966372 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:07:31.966439 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.966411 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:07:31.966662 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.966444 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:32.966424875 +0000 UTC m=+37.411909983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : configmap "kube-rbac-proxy" not found Jul 12 08:07:31.966662 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:31.966464 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:32.966455161 +0000 UTC m=+37.411940270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : secret "insights-runtime-extractor-tls" not found Jul 12 08:07:32.167841 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.167762 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:32.167976 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:32.167872 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:07:32.167976 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:32.167939 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls podName:8148b2db-3b6c-47ed-ba45-fcd50dd4d57d nodeName:}" failed. No retries permitted until 2026-07-12 08:07:34.167920328 +0000 UTC m=+38.613405441 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls") pod "dns-default-grhbr" (UID: "8148b2db-3b6c-47ed-ba45-fcd50dd4d57d") : secret "dns-default-metrics-tls" not found Jul 12 08:07:32.284526 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.284489 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:32.332483 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.332438 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:32.522732 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.522648 2558 generic.go:356] "Generic (PLEG): container finished" podID="fa000e92-7ae2-4662-aa9f-11a3c2ff42cd" containerID="4a55accd9eccbf85168e2e20492ad7a8a9d3110d4fe561e37324a5e79b9abbb5" exitCode=0 Jul 12 08:07:32.522732 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.522697 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerDied","Data":"4a55accd9eccbf85168e2e20492ad7a8a9d3110d4fe561e37324a5e79b9abbb5"} Jul 12 08:07:32.524122 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.524052 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l8z9l" event={"ID":"09bc5f43-1026-4cdc-8a41-eae824bd5d7c","Type":"ContainerStarted","Data":"6fb0af3f43895eba7f6f60fad93e5673636b21d44e21bc0c6ebdd576fdfff2a7"} Jul 12 08:07:32.631524 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.631502 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts"] Jul 12 08:07:32.638108 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.638086 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" Jul 12 08:07:32.640365 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.640273 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 08:07:32.640479 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.640436 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 08:07:32.640531 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.640485 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fvz2v\"" Jul 12 08:07:32.643232 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.643212 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts"] Jul 12 08:07:32.669140 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.669111 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:32.763655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.763627 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:32.771708 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.771683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msxj2\" (UniqueName: \"kubernetes.io/projected/4e3059d4-f150-4292-9844-224920c3ac11-kube-api-access-msxj2\") pod \"migrator-7f59b54f6c-22zts\" (UID: \"4e3059d4-f150-4292-9844-224920c3ac11\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" Jul 12 08:07:32.872987 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.872953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msxj2\" (UniqueName: \"kubernetes.io/projected/4e3059d4-f150-4292-9844-224920c3ac11-kube-api-access-msxj2\") pod \"migrator-7f59b54f6c-22zts\" (UID: \"4e3059d4-f150-4292-9844-224920c3ac11\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" Jul 12 08:07:32.881045 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.881020 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msxj2\" (UniqueName: \"kubernetes.io/projected/4e3059d4-f150-4292-9844-224920c3ac11-kube-api-access-msxj2\") pod \"migrator-7f59b54f6c-22zts\" (UID: \"4e3059d4-f150-4292-9844-224920c3ac11\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" Jul 12 08:07:32.974263 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.974223 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:32.974263 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.974268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:32.974471 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:32.974357 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:07:32.974471 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:32.974375 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:07:32.974471 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:32.974425 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:34.97440727 +0000 UTC m=+39.419892384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : secret "insights-runtime-extractor-tls" not found Jul 12 08:07:32.974471 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:32.974442 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:34.974433188 +0000 UTC m=+39.419918297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : configmap "kube-rbac-proxy" not found Jul 12 08:07:32.998241 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:32.998215 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" Jul 12 08:07:33.369074 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.369042 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-7vwdp"] Jul 12 08:07:33.372117 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.372095 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.375375 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.375356 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 08:07:33.375511 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.375494 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ntl9z\"" Jul 12 08:07:33.376367 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.376347 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 08:07:33.376367 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.376361 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 08:07:33.376539 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.376418 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 08:07:33.383884 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.383865 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-7vwdp"] Jul 12 08:07:33.414640 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.414608 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:33.421999 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.421979 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zgdp2_3a49d418-33d8-4566-aed2-dad75f7401bb/dns-node-resolver/0.log" Jul 12 08:07:33.463167 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.463115 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:33.478306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.478271 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a91db01b-ecaf-4eff-8d42-9835d0670ea0-signing-key\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.478450 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.478414 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw5s9\" (UniqueName: \"kubernetes.io/projected/a91db01b-ecaf-4eff-8d42-9835d0670ea0-kube-api-access-bw5s9\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.478551 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.478501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a91db01b-ecaf-4eff-8d42-9835d0670ea0-signing-cabundle\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.531517 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.531484 2558 generic.go:356] "Generic (PLEG): container finished" podID="fa000e92-7ae2-4662-aa9f-11a3c2ff42cd" containerID="82b27931e15438b5a8abe901d04a35d0eeb098e23c61bb2ff283f2a28e8186e9" exitCode=0 Jul 12 08:07:33.531649 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.531553 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerDied","Data":"82b27931e15438b5a8abe901d04a35d0eeb098e23c61bb2ff283f2a28e8186e9"} Jul 12 08:07:33.594260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.592064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw5s9\" (UniqueName: \"kubernetes.io/projected/a91db01b-ecaf-4eff-8d42-9835d0670ea0-kube-api-access-bw5s9\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.594260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.592148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a91db01b-ecaf-4eff-8d42-9835d0670ea0-signing-cabundle\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.594260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.592195 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a91db01b-ecaf-4eff-8d42-9835d0670ea0-signing-key\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.594260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.594021 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a91db01b-ecaf-4eff-8d42-9835d0670ea0-signing-cabundle\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.604110 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.600280 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a91db01b-ecaf-4eff-8d42-9835d0670ea0-signing-key\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.604110 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.602689 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw5s9\" (UniqueName: \"kubernetes.io/projected/a91db01b-ecaf-4eff-8d42-9835d0670ea0-kube-api-access-bw5s9\") pod \"service-ca-79f68b9f4f-7vwdp\" (UID: \"a91db01b-ecaf-4eff-8d42-9835d0670ea0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.665550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.665525 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts"] Jul 12 08:07:33.672088 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:33.672066 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3059d4_f150_4292_9844_224920c3ac11.slice/crio-3d551c0a79bc941fb62f50be63301973aa3ed09678c168da8ffb69407a25e99a WatchSource:0}: Error finding container 3d551c0a79bc941fb62f50be63301973aa3ed09678c168da8ffb69407a25e99a: Status 404 returned error can't find the container with id 3d551c0a79bc941fb62f50be63301973aa3ed09678c168da8ffb69407a25e99a Jul 12 08:07:33.682359 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.682338 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" Jul 12 08:07:33.831282 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.831247 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-7vwdp"] Jul 12 08:07:33.834404 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:33.834378 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91db01b_ecaf_4eff_8d42_9835d0670ea0.slice/crio-f43c51939f9cf4fcc8fab4fc4e0a90917784159462e50aee0d7cb77a6b61c665 WatchSource:0}: Error finding container f43c51939f9cf4fcc8fab4fc4e0a90917784159462e50aee0d7cb77a6b61c665: Status 404 returned error can't find the container with id f43c51939f9cf4fcc8fab4fc4e0a90917784159462e50aee0d7cb77a6b61c665 Jul 12 08:07:33.906760 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:33.906637 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:34.197466 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.197382 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:34.197595 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:34.197500 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:07:34.197595 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:34.197551 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls podName:8148b2db-3b6c-47ed-ba45-fcd50dd4d57d nodeName:}" failed. No retries permitted until 2026-07-12 08:07:38.197537975 +0000 UTC m=+42.643023086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls") pod "dns-default-grhbr" (UID: "8148b2db-3b6c-47ed-ba45-fcd50dd4d57d") : secret "dns-default-metrics-tls" not found Jul 12 08:07:34.233893 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.233864 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lnw96_902ddd8c-1f3c-423f-bb33-498fcd687380/node-ca/0.log" Jul 12 08:07:34.258370 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.258335 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:34.538795 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.538664 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29ggz" event={"ID":"fa000e92-7ae2-4662-aa9f-11a3c2ff42cd","Type":"ContainerStarted","Data":"4a4d29358d62809c60120e9c37cf32e24e1095b6f66a9df9b14d6ad55103615e"} Jul 12 08:07:34.540951 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.540921 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l8z9l" event={"ID":"09bc5f43-1026-4cdc-8a41-eae824bd5d7c","Type":"ContainerStarted","Data":"f1a4edb06e8d4d2489325563840e78ecbca8060a1f902045b2315dddfaa09638"} Jul 12 08:07:34.542715 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.542676 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" event={"ID":"4e3059d4-f150-4292-9844-224920c3ac11","Type":"ContainerStarted","Data":"3d551c0a79bc941fb62f50be63301973aa3ed09678c168da8ffb69407a25e99a"} Jul 12 08:07:34.544538 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.544479 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" event={"ID":"a91db01b-ecaf-4eff-8d42-9835d0670ea0","Type":"ContainerStarted","Data":"f43c51939f9cf4fcc8fab4fc4e0a90917784159462e50aee0d7cb77a6b61c665"} Jul 12 08:07:34.563612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.563551 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-29ggz" podStartSLOduration=8.544097757 podStartE2EDuration="38.56353225s" totalImagesPullingTime="30.019434493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:34.561673062 +0000 UTC m=+39.007158207" watchObservedRunningTime="2026-07-12 08:07:34.56353225 +0000 UTC m=+39.009017361" Jul 12 08:07:34.577092 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.577040 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l8z9l" podStartSLOduration=2.538790809 podStartE2EDuration="4.577012976s" totalImagesPullingTime="2.038222167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:34.576206496 +0000 UTC m=+39.021691627" watchObservedRunningTime="2026-07-12 08:07:34.577012976 +0000 UTC m=+39.022498105" Jul 12 08:07:34.822332 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:34.822307 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l8z9l_09bc5f43-1026-4cdc-8a41-eae824bd5d7c/serve-healthcheck-canary/0.log" Jul 12 08:07:35.006096 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:35.005663 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:35.006096 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:35.006054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:35.006616 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:35.005783 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:07:35.006616 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:35.006177 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:07:35.006616 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:35.006245 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:39.006226994 +0000 UTC m=+43.451712103 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : configmap "kube-rbac-proxy" not found Jul 12 08:07:35.006616 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:35.006326 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:39.00630449 +0000 UTC m=+43.451789601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : secret "insights-runtime-extractor-tls" not found Jul 12 08:07:35.578709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:35.578679 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:35.745174 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:35.745040 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:36.552832 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:36.552801 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" event={"ID":"4e3059d4-f150-4292-9844-224920c3ac11","Type":"ContainerStarted","Data":"80c114c9ce18bb890539a6e19f13a3720fbc49acdd9fd8b6cc2579696bc22219"} Jul 12 08:07:36.553235 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:36.552860 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" event={"ID":"4e3059d4-f150-4292-9844-224920c3ac11","Type":"ContainerStarted","Data":"e7e6df92b669cf1b7bf0dbce0a8bba6922521a651cd3ebed84a7c65301ff3899"} Jul 12 08:07:36.567577 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:36.567537 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-22zts" podStartSLOduration=2.765330412 podStartE2EDuration="4.567520897s" totalImagesPullingTime="1.802190485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:36.566516504 +0000 UTC m=+41.012001635" watchObservedRunningTime="2026-07-12 08:07:36.567520897 +0000 UTC m=+41.013006027" Jul 12 08:07:37.286212 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:37.286178 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:37.556568 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:37.556481 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" event={"ID":"a91db01b-ecaf-4eff-8d42-9835d0670ea0","Type":"ContainerStarted","Data":"851a6febbc1ab7c8eddc598160c79a8f821dcb8d7169360d990875819be5c46c"} Jul 12 08:07:37.574468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:37.574421 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-7vwdp" podStartSLOduration=1.9512466819999998 podStartE2EDuration="4.574401875s" totalImagesPullingTime="2.623155193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:37.57407754 +0000 UTC m=+42.019562681" watchObservedRunningTime="2026-07-12 08:07:37.574401875 +0000 UTC m=+42.019887005" Jul 12 08:07:38.233024 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:38.232982 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:38.233198 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:38.233170 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:07:38.233268 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:38.233256 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls podName:8148b2db-3b6c-47ed-ba45-fcd50dd4d57d nodeName:}" failed. No retries permitted until 2026-07-12 08:07:46.233234011 +0000 UTC m=+50.678719121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls") pod "dns-default-grhbr" (UID: "8148b2db-3b6c-47ed-ba45-fcd50dd4d57d") : secret "dns-default-metrics-tls" not found Jul 12 08:07:38.663363 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:38.663318 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:38.861963 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:38.861924 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:38.963446 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:38.963360 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:39.039637 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:39.039592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:39.039845 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:39.039652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:39.039845 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:39.039789 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:07:39.040367 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:39.040347 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6138865-5794-481a-a099-18afed93496b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:39.040439 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:07:39.040387 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls podName:f6138865-5794-481a-a099-18afed93496b nodeName:}" failed. No retries permitted until 2026-07-12 08:07:47.040365341 +0000 UTC m=+51.485850465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g25qv" (UID: "f6138865-5794-481a-a099-18afed93496b") : secret "insights-runtime-extractor-tls" not found Jul 12 08:07:39.120109 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:39.120059 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:39.220313 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:39.220214 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:39.358311 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:39.358278 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:40.157689 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:40.157656 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:40.556441 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:40.556361 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:40.613853 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:40.613818 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:40.759338 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:40.759304 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:41.360721 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:41.360683 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:41.954431 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:41.954380 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:42.157260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:42.157204 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:42.357620 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:42.357586 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:42.410565 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:42.410529 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:42.750196 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:42.750113 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:42.801583 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:42.801558 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:43.156738 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:43.156703 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:43.209394 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:43.209366 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:43.958260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:43.958228 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:44.159283 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:44.159249 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:44.356014 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:44.355980 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:44.759904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:44.759831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:44.972754 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:44.972725 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:45.559755 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:45.559720 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:45.754971 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:45.754936 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:45.958510 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:45.958436 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:46.296131 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.296047 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:46.298930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.298903 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8148b2db-3b6c-47ed-ba45-fcd50dd4d57d-metrics-tls\") pod \"dns-default-grhbr\" (UID: \"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d\") " pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:46.349191 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.349171 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:46.356470 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.356446 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:46.486891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.486860 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-grhbr"] Jul 12 08:07:46.489804 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:46.489780 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8148b2db_3b6c_47ed_ba45_fcd50dd4d57d.slice/crio-ce7766304db518f9992e1a983cd459d581c1eecbda125fb6c3538a1cb5b0917e WatchSource:0}: Error finding container ce7766304db518f9992e1a983cd459d581c1eecbda125fb6c3538a1cb5b0917e: Status 404 returned error can't find the container with id ce7766304db518f9992e1a983cd459d581c1eecbda125fb6c3538a1cb5b0917e Jul 12 08:07:46.579260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.579232 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-grhbr" event={"ID":"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d","Type":"ContainerStarted","Data":"ce7766304db518f9992e1a983cd459d581c1eecbda125fb6c3538a1cb5b0917e"} Jul 12 08:07:46.955917 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:46.955831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:47.101787 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.101736 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:47.104090 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.104057 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6138865-5794-481a-a099-18afed93496b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g25qv\" (UID: \"f6138865-5794-481a-a099-18afed93496b\") " pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:47.156953 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.156922 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g25qv" Jul 12 08:07:47.307019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.306985 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g25qv"] Jul 12 08:07:47.310196 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:47.310162 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6138865_5794_481a_a099_18afed93496b.slice/crio-ca33821bb1202b2d0cfa6a87af18dd92741b1bb4cc3a7992cfcd5475e0676218 WatchSource:0}: Error finding container ca33821bb1202b2d0cfa6a87af18dd92741b1bb4cc3a7992cfcd5475e0676218: Status 404 returned error can't find the container with id ca33821bb1202b2d0cfa6a87af18dd92741b1bb4cc3a7992cfcd5475e0676218 Jul 12 08:07:47.355497 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.355458 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:47.583656 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.583622 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g25qv" event={"ID":"f6138865-5794-481a-a099-18afed93496b","Type":"ContainerStarted","Data":"ca33821bb1202b2d0cfa6a87af18dd92741b1bb4cc3a7992cfcd5475e0676218"} Jul 12 08:07:47.758673 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.758641 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:47.958696 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:47.958613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:48.362649 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:48.362556 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:49.006811 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:49.006784 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:49.157063 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:49.157031 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:49.353912 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:49.353878 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:49.591472 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:49.591440 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-grhbr" event={"ID":"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d","Type":"ContainerStarted","Data":"1a224961a067d4c759b6430fcdd9eb08b08ef399f3d52d8080b48172b69ebabb"} Jul 12 08:07:51.602524 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:51.602490 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g25qv" event={"ID":"f6138865-5794-481a-a099-18afed93496b","Type":"ContainerStarted","Data":"4e13128dc426411bf9df22817158089668e0fb23f89a000661036bd069dad25b"} Jul 12 08:07:51.604121 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:51.604099 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-grhbr" event={"ID":"8148b2db-3b6c-47ed-ba45-fcd50dd4d57d","Type":"ContainerStarted","Data":"f66d02d4f713277b73c8432af9f09628f2faa5d7187365ef6464ce58287196a6"} Jul 12 08:07:51.604355 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:51.604334 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-grhbr" Jul 12 08:07:51.620019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:51.619980 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-grhbr" podStartSLOduration=19.17053606 podStartE2EDuration="21.619968024s" totalImagesPullingTime="2.449431964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:51.61967712 +0000 UTC m=+56.065162251" watchObservedRunningTime="2026-07-12 08:07:51.619968024 +0000 UTC m=+56.065453153" Jul 12 08:07:52.608874 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:52.608827 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g25qv" event={"ID":"f6138865-5794-481a-a099-18afed93496b","Type":"ContainerStarted","Data":"c3653f728af47f266b168b40b99a935e699c3914309ffda7d7e114975e32bf35"} Jul 12 08:07:54.617425 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:54.617341 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g25qv" event={"ID":"f6138865-5794-481a-a099-18afed93496b","Type":"ContainerStarted","Data":"e5a2c87d831f81be515012e3fc594383e2429b1616f721ae7054127dd4651207"} Jul 12 08:07:54.632954 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:54.632911 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g25qv" podStartSLOduration=16.717971688 podStartE2EDuration="23.632896059s" totalImagesPullingTime="6.914924371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:07:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:07:54.632109896 +0000 UTC m=+59.077595054" watchObservedRunningTime="2026-07-12 08:07:54.632896059 +0000 UTC m=+59.078381179" Jul 12 08:07:56.518399 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:56.518366 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chzp5" Jul 12 08:07:57.001557 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.001527 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk"] Jul 12 08:07:57.004073 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.004049 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.006669 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.006645 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 12 08:07:57.006812 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.006794 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 08:07:57.007531 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.007508 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 08:07:57.007655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.007547 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 08:07:57.009709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.009687 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4"] Jul 12 08:07:57.015435 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.015403 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.017459 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.017441 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 12 08:07:57.017678 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.017664 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 12 08:07:57.017866 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.017829 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 12 08:07:57.017938 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.017928 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 12 08:07:57.020226 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.020185 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk"] Jul 12 08:07:57.024075 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.024052 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4"] Jul 12 08:07:57.082116 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082090 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.082299 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082125 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.082299 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082195 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8327520d-5c45-416c-aa72-32fed1402dcc-tmp\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.082299 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-ca\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.082299 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-hub\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.082299 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082280 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8327520d-5c45-416c-aa72-32fed1402dcc-klusterlet-config\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.082468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082343 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4fkt\" (UniqueName: \"kubernetes.io/projected/8327520d-5c45-416c-aa72-32fed1402dcc-kube-api-access-b4fkt\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.082468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082421 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d210730-36aa-4e28-8e10-141e83717b23-additional-service-ca\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.082468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.082439 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfgcl\" (UniqueName: \"kubernetes.io/projected/0d210730-36aa-4e28-8e10-141e83717b23-kube-api-access-kfgcl\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.110218 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.110188 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:57.182985 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.182956 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d210730-36aa-4e28-8e10-141e83717b23-additional-service-ca\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.183154 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.182989 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfgcl\" (UniqueName: \"kubernetes.io/projected/0d210730-36aa-4e28-8e10-141e83717b23-kube-api-access-kfgcl\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.183154 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183014 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.183154 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.183154 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183121 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8327520d-5c45-416c-aa72-32fed1402dcc-tmp\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.183355 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-ca\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.183401 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-hub\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.183457 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183423 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8327520d-5c45-416c-aa72-32fed1402dcc-klusterlet-config\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.183509 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4fkt\" (UniqueName: \"kubernetes.io/projected/8327520d-5c45-416c-aa72-32fed1402dcc-kube-api-access-b4fkt\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.183509 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183499 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8327520d-5c45-416c-aa72-32fed1402dcc-tmp\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.183788 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.183735 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d210730-36aa-4e28-8e10-141e83717b23-additional-service-ca\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.185809 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.185785 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-ca\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.185941 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.185923 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.186035 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.186013 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-hub\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.186125 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.186105 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0d210730-36aa-4e28-8e10-141e83717b23-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.186342 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.186322 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8327520d-5c45-416c-aa72-32fed1402dcc-klusterlet-config\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.202087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.202062 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4fkt\" (UniqueName: \"kubernetes.io/projected/8327520d-5c45-416c-aa72-32fed1402dcc-kube-api-access-b4fkt\") pod \"klusterlet-addon-workmgr-85fb5dd54b-bkbkk\" (UID: \"8327520d-5c45-416c-aa72-32fed1402dcc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.202202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.202186 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfgcl\" (UniqueName: \"kubernetes.io/projected/0d210730-36aa-4e28-8e10-141e83717b23-kube-api-access-kfgcl\") pod \"cluster-proxy-proxy-agent-568457c747-rgnv4\" (UID: \"0d210730-36aa-4e28-8e10-141e83717b23\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.211487 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.211464 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:57.302954 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.302853 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:57.315703 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.315674 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:07:57.334609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.334589 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" Jul 12 08:07:57.403268 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.403240 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:57.447481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.447455 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk"] Jul 12 08:07:57.449956 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:57.449922 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8327520d_5c45_416c_aa72_32fed1402dcc.slice/crio-6071f448ccae5ca1faed41da2a2fa2395bd9f1dd2a921e25dbe31eb9d50238cb WatchSource:0}: Error finding container 6071f448ccae5ca1faed41da2a2fa2395bd9f1dd2a921e25dbe31eb9d50238cb: Status 404 returned error can't find the container with id 6071f448ccae5ca1faed41da2a2fa2395bd9f1dd2a921e25dbe31eb9d50238cb Jul 12 08:07:57.463590 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.463568 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4"] Jul 12 08:07:57.466445 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:07:57.466419 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d210730_36aa_4e28_8e10_141e83717b23.slice/crio-3e4dccd9618009d3f4b9135d55dc7b84fef5808cea2f9e3ab96675bffde6ef59 WatchSource:0}: Error finding container 3e4dccd9618009d3f4b9135d55dc7b84fef5808cea2f9e3ab96675bffde6ef59: Status 404 returned error can't find the container with id 3e4dccd9618009d3f4b9135d55dc7b84fef5808cea2f9e3ab96675bffde6ef59 Jul 12 08:07:57.630156 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.630121 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" event={"ID":"0d210730-36aa-4e28-8e10-141e83717b23","Type":"ContainerStarted","Data":"3e4dccd9618009d3f4b9135d55dc7b84fef5808cea2f9e3ab96675bffde6ef59"} Jul 12 08:07:57.631225 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.631201 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" event={"ID":"8327520d-5c45-416c-aa72-32fed1402dcc","Type":"ContainerStarted","Data":"6071f448ccae5ca1faed41da2a2fa2395bd9f1dd2a921e25dbe31eb9d50238cb"} Jul 12 08:07:57.754996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.754965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:57.810950 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:57.810902 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:58.856017 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:58.855985 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:58.959870 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:58.959824 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.047616 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.047583 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.422155 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.422120 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.520092 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.520020 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.591495 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.591421 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.663523 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.661795 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.786570 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.786458 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.887351 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.887285 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:07:59.942649 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:07:59.942593 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.025323 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.025269 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.126229 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.126189 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.226563 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.226527 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.280880 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.280835 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.441719 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.441641 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.491422 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.491386 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.608457 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.608425 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:00.658930 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:00.658897 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:01.450939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.450905 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:01.611607 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.611579 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-grhbr" Jul 12 08:08:01.619497 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.619393 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:01.717653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.717557 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:01.828073 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.828029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:08:01.830518 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.830491 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 08:08:01.840444 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.840419 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 08:08:01.852041 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.852015 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29l9x\" (UniqueName: \"kubernetes.io/projected/706842c4-d3bb-42f3-8168-db36876a85c9-kube-api-access-29l9x\") pod \"network-check-target-xsdxv\" (UID: \"706842c4-d3bb-42f3-8168-db36876a85c9\") " pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:08:01.896389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.896350 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:01.911503 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.911476 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xxbfq\"" Jul 12 08:08:01.920299 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.920273 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:08:01.929391 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.929369 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:08:01.929504 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.929448 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:08:01.931609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.931587 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 08:08:01.931609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.931611 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 08:08:01.943152 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.943130 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/083f72d9-be4f-4fcb-a2a3-19fd4c746884-original-pull-secret\") pod \"global-pull-secret-syncer-8gf2t\" (UID: \"083f72d9-be4f-4fcb-a2a3-19fd4c746884\") " pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:08:01.943238 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.943223 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fe07317-22f5-4710-9349-22d9672475e1-metrics-certs\") pod \"network-metrics-daemon-gjm7q\" (UID: \"1fe07317-22f5-4710-9349-22d9672475e1\") " pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:08:01.946549 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:01.946520 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:02.017147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:02.017063 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:02.116707 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:02.116673 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:02.199186 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:02.199159 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rkfrj\"" Jul 12 08:08:02.204066 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:02.204044 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gf2t" Jul 12 08:08:02.206932 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:02.206909 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjm7q" Jul 12 08:08:02.273179 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:02.273100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:03.304346 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.304316 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xsdxv"] Jul 12 08:08:03.308880 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:08:03.308853 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706842c4_d3bb_42f3_8168_db36876a85c9.slice/crio-b77da6385bcb1d0b243ef0270bfa30b8621de4c5be10e02b3aa518f4531f0be4 WatchSource:0}: Error finding container b77da6385bcb1d0b243ef0270bfa30b8621de4c5be10e02b3aa518f4531f0be4: Status 404 returned error can't find the container with id b77da6385bcb1d0b243ef0270bfa30b8621de4c5be10e02b3aa518f4531f0be4 Jul 12 08:08:03.527436 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.527410 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8gf2t"] Jul 12 08:08:03.530014 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:08:03.529988 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083f72d9_be4f_4fcb_a2a3_19fd4c746884.slice/crio-364fa05cf91a42f01ec9d810eebe92f55de7ddc5220b50093ae8354416def18b WatchSource:0}: Error finding container 364fa05cf91a42f01ec9d810eebe92f55de7ddc5220b50093ae8354416def18b: Status 404 returned error can't find the container with id 364fa05cf91a42f01ec9d810eebe92f55de7ddc5220b50093ae8354416def18b Jul 12 08:08:03.530521 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.530503 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gjm7q"] Jul 12 08:08:03.533493 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:08:03.533474 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe07317_22f5_4710_9349_22d9672475e1.slice/crio-eee4e71d7a12e8de152fac3d823b809ed849c43eeba75b9371502b0d401e5b15 WatchSource:0}: Error finding container eee4e71d7a12e8de152fac3d823b809ed849c43eeba75b9371502b0d401e5b15: Status 404 returned error can't find the container with id eee4e71d7a12e8de152fac3d823b809ed849c43eeba75b9371502b0d401e5b15 Jul 12 08:08:03.656141 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.656056 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" event={"ID":"8327520d-5c45-416c-aa72-32fed1402dcc","Type":"ContainerStarted","Data":"b0aee37440baa4d3af75db6ebde35462c9a7956bacb11eea9baf14a5b46e1d37"} Jul 12 08:08:03.656294 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.656238 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:08:03.657546 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.657517 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8gf2t" event={"ID":"083f72d9-be4f-4fcb-a2a3-19fd4c746884","Type":"ContainerStarted","Data":"364fa05cf91a42f01ec9d810eebe92f55de7ddc5220b50093ae8354416def18b"} Jul 12 08:08:03.658184 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.658152 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" Jul 12 08:08:03.659756 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.659718 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" event={"ID":"0d210730-36aa-4e28-8e10-141e83717b23","Type":"ContainerStarted","Data":"a39ec8bd80108d0eec53b169a111106872c66bddf29780a5d787a35f1c466927"} Jul 12 08:08:03.659850 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.659776 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" event={"ID":"0d210730-36aa-4e28-8e10-141e83717b23","Type":"ContainerStarted","Data":"5461c6b06f03b078e9a2e846a7c89728f1f28b374f660e06d44fb5a1fe0196a7"} Jul 12 08:08:03.659850 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.659797 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" event={"ID":"0d210730-36aa-4e28-8e10-141e83717b23","Type":"ContainerStarted","Data":"89dfa20ed72f37c1c4bb20736d6b8373723440cba44b93a95d16aefbb94a5106"} Jul 12 08:08:03.660932 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.660912 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gjm7q" event={"ID":"1fe07317-22f5-4710-9349-22d9672475e1","Type":"ContainerStarted","Data":"eee4e71d7a12e8de152fac3d823b809ed849c43eeba75b9371502b0d401e5b15"} Jul 12 08:08:03.661953 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.661935 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xsdxv" event={"ID":"706842c4-d3bb-42f3-8168-db36876a85c9","Type":"ContainerStarted","Data":"b77da6385bcb1d0b243ef0270bfa30b8621de4c5be10e02b3aa518f4531f0be4"} Jul 12 08:08:03.672231 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.672189 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85fb5dd54b-bkbkk" podStartSLOduration=1.960746537 podStartE2EDuration="7.672177972s" totalImagesPullingTime="5.711431435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:08:03.670932602 +0000 UTC m=+68.116417732" watchObservedRunningTime="2026-07-12 08:08:03.672177972 +0000 UTC m=+68.117663102" Jul 12 08:08:03.701852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:03.701787 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-568457c747-rgnv4" podStartSLOduration=1.9921050089999999 podStartE2EDuration="7.701774745s" totalImagesPullingTime="5.709669736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:08:03.701263527 +0000 UTC m=+68.146748658" watchObservedRunningTime="2026-07-12 08:08:03.701774745 +0000 UTC m=+68.147259876" Jul 12 08:08:04.030265 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:04.030152 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:04.090941 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:04.090878 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:04.884688 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:04.884644 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:05.673779 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:05.673675 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gjm7q" event={"ID":"1fe07317-22f5-4710-9349-22d9672475e1","Type":"ContainerStarted","Data":"41f9d3fd0609c9714b8fa73658506b61a4e1a21fb38e796a2c759cf7552a1f16"} Jul 12 08:08:05.673941 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:05.673814 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gjm7q" event={"ID":"1fe07317-22f5-4710-9349-22d9672475e1","Type":"ContainerStarted","Data":"2d4bbc4d1a0204db5779d3e5a418ab09452afc970e8a3c68b292f30bf1f5bc27"} Jul 12 08:08:05.690634 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:05.690579 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gjm7q" podStartSLOduration=68.064515315 podStartE2EDuration="1m9.690560822s" totalImagesPullingTime="1.626045507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:08:05.688658159 +0000 UTC m=+70.134143291" watchObservedRunningTime="2026-07-12 08:08:05.690560822 +0000 UTC m=+70.136045953" Jul 12 08:08:05.781475 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:05.781439 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:05.879243 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:05.879139 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:06.358119 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:06.358086 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:06.449914 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:06.449880 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:07.693197 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:07.693165 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:08.687693 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:08.687645 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8gf2t" event={"ID":"083f72d9-be4f-4fcb-a2a3-19fd4c746884","Type":"ContainerStarted","Data":"a07e36e25512724efd35c9fb55e69ac20506c3bbb480bee8cfdf65df6c0100cc"} Jul 12 08:08:08.689106 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:08.689080 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xsdxv" event={"ID":"706842c4-d3bb-42f3-8168-db36876a85c9","Type":"ContainerStarted","Data":"fd894680fabb2fa03bd2dd514d295f3af07a5424776693f39845a74173cd9f81"} Jul 12 08:08:08.689227 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:08.689132 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:08:08.703102 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:08.703063 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8gf2t" podStartSLOduration=68.231073829 podStartE2EDuration="1m12.703050709s" totalImagesPullingTime="4.47197688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:08:08.702575907 +0000 UTC m=+73.148061036" watchObservedRunningTime="2026-07-12 08:08:08.703050709 +0000 UTC m=+73.148535838" Jul 12 08:08:08.715670 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:08.715625 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xsdxv" podStartSLOduration=68.028377928 podStartE2EDuration="1m12.71561351s" totalImagesPullingTime="4.687235582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:08:08.715035783 +0000 UTC m=+73.160520913" watchObservedRunningTime="2026-07-12 08:08:08.71561351 +0000 UTC m=+73.161098655" Jul 12 08:08:10.192972 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.192936 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.287293 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.287265 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.412384 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.412355 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.484864 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.484793 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kgbnt"] Jul 12 08:08:10.488449 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.488432 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.490689 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.490667 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 08:08:10.490883 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.490857 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 08:08:10.491002 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.490985 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 08:08:10.491085 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.491070 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 08:08:10.491145 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.491119 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 08:08:10.491684 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.491666 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 08:08:10.491814 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.491717 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jwfft\"" Jul 12 08:08:10.500562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.500539 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.598007 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.597980 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-wtmp\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598143 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598021 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598143 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-accelerators-collector-config\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598238 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lq6\" (UniqueName: \"kubernetes.io/projected/9af93d0b-cb4d-4086-bb54-731f1c277eff-kube-api-access-s5lq6\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598238 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598203 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-sys\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598245 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af93d0b-cb4d-4086-bb54-731f1c277eff-metrics-client-ca\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598286 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-textfile\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598329 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-root\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.598366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.598352 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-tls\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.677275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.677251 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.698629 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lq6\" (UniqueName: \"kubernetes.io/projected/9af93d0b-cb4d-4086-bb54-731f1c277eff-kube-api-access-s5lq6\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.698772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698637 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-sys\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.698772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698655 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af93d0b-cb4d-4086-bb54-731f1c277eff-metrics-client-ca\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.698772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698672 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-textfile\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.698772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698692 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-root\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.698772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698714 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-tls\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.698772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698735 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-sys\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698839 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-root\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698854 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-wtmp\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698886 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.698930 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-accelerators-collector-config\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.699031 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-wtmp\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.699042 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-textfile\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699344 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.699325 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af93d0b-cb4d-4086-bb54-731f1c277eff-metrics-client-ca\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.699401 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.699386 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-accelerators-collector-config\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.701453 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.701435 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.701575 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.701557 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9af93d0b-cb4d-4086-bb54-731f1c277eff-node-exporter-tls\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.706198 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.706169 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lq6\" (UniqueName: \"kubernetes.io/projected/9af93d0b-cb4d-4086-bb54-731f1c277eff-kube-api-access-s5lq6\") pod \"node-exporter-kgbnt\" (UID: \"9af93d0b-cb4d-4086-bb54-731f1c277eff\") " pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.778714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.778658 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.798365 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.798346 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kgbnt" Jul 12 08:08:10.809029 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:08:10.809008 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af93d0b_cb4d_4086_bb54_731f1c277eff.slice/crio-b441a6d3db91c61e9fbe40643cf917efea5ee94fa19c582ab69b1fcc17e43dc4 WatchSource:0}: Error finding container b441a6d3db91c61e9fbe40643cf917efea5ee94fa19c582ab69b1fcc17e43dc4: Status 404 returned error can't find the container with id b441a6d3db91c61e9fbe40643cf917efea5ee94fa19c582ab69b1fcc17e43dc4 Jul 12 08:08:10.890407 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.890376 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:10.991231 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:10.991196 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:11.092707 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:11.092672 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:11.192076 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:11.192042 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:11.403356 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:11.403259 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:11.490391 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:11.490358 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:11.650238 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:11.650198 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:11.701763 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:11.701683 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgbnt" event={"ID":"9af93d0b-cb4d-4086-bb54-731f1c277eff","Type":"ContainerStarted","Data":"b441a6d3db91c61e9fbe40643cf917efea5ee94fa19c582ab69b1fcc17e43dc4"} Jul 12 08:08:12.708896 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:12.708862 2558 generic.go:356] "Generic (PLEG): container finished" podID="9af93d0b-cb4d-4086-bb54-731f1c277eff" containerID="ee60796517d3d8d7aa18fe75d5517cc81efc946f140b8b1abe7520ca5196a490" exitCode=0 Jul 12 08:08:12.708896 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:12.708902 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgbnt" event={"ID":"9af93d0b-cb4d-4086-bb54-731f1c277eff","Type":"ContainerDied","Data":"ee60796517d3d8d7aa18fe75d5517cc81efc946f140b8b1abe7520ca5196a490"} Jul 12 08:08:12.752280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:12.752253 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:12.804856 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:12.804830 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:13.348560 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:13.348523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:13.419391 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:13.419357 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:13.591214 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:13.591179 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:13.691295 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:13.691208 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:13.713872 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:13.713840 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgbnt" event={"ID":"9af93d0b-cb4d-4086-bb54-731f1c277eff","Type":"ContainerStarted","Data":"ca9982e58cb9e71decfcee365833a8500af759e4949e3050a5429a9c6245d161"} Jul 12 08:08:13.714244 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:13.713897 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgbnt" event={"ID":"9af93d0b-cb4d-4086-bb54-731f1c277eff","Type":"ContainerStarted","Data":"18833af6e761830261f80cd0c194fa120f42cd956b424dde22d57636400fd2ec"} Jul 12 08:08:15.072035 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:15.072002 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:15.174167 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:15.174138 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:15.227841 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:15.227815 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:15.373295 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:15.373215 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:15.474412 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:15.474383 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:15.565679 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:15.565651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:16.605327 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:16.605292 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:19.373536 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:19.373415 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:19.466124 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:19.466095 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:20.882518 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:20.882483 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:20.930191 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:20.930170 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:24.118687 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:24.118655 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:24.165788 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:24.165760 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:24.274716 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:24.274687 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:24.375019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:24.374954 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:24.429152 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:24.429126 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:26.286898 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:26.286865 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:26.344569 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:26.344545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:27.274262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:27.274228 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:27.324085 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:27.324053 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:27.754089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:27.754055 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:27.804770 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:27.804724 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:29.642844 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:29.642812 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:29.698891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:29.698867 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:30.507137 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:30.507100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:32.719955 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:32.719920 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:32.766137 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:32.766113 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:39.694780 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:39.694728 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xsdxv" Jul 12 08:08:39.708576 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:39.708525 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kgbnt" podStartSLOduration=28.731519752 podStartE2EDuration="29.708511462s" totalImagesPullingTime="913.828113ms" totalInitContainerRuntime="63.163597ms" isStatefulPod=true podCreationTimestamp="2026-07-12 08:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:08:13.731693551 +0000 UTC m=+78.177178681" watchObservedRunningTime="2026-07-12 08:08:39.708511462 +0000 UTC m=+104.153996656" Jul 12 08:08:42.761298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:42.761263 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:42.860147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:42.860112 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:49.928696 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:49.928663 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:57.166662 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:57.166637 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l8z9l_09bc5f43-1026-4cdc-8a41-eae824bd5d7c/serve-healthcheck-canary/0.log" Jul 12 08:08:57.763198 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:57.763163 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:08:57.815779 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:08:57.815732 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:05.154793 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:05.154757 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:05.258835 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:05.255012 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:05.301593 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:05.301552 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:29.611455 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:29.611423 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:29.991721 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:29.991646 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:38.396369 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:38.396334 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:38.496820 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:38.496791 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:48.950405 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:48.950318 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:49.030360 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:49.030328 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:09:49.081017 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:09:49.080986 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:19.669322 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:19.669288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:27.265837 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:27.265801 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:44.823999 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:44.823961 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:44.924443 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:44.924412 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:49.379903 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:49.379867 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:49.431601 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:49.431552 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:49.527952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:49.527921 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:49.577354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:49.577326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:50.125076 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:50.125046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:50.178530 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:50.178505 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:56.825779 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:56.825732 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:10:56.926805 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:10:56.926771 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:00.197335 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:00.197301 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:00.243665 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:00.243617 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:00.400831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:00.400789 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:00.502084 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:00.501996 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:12.762183 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:12.762146 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:12.861414 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:12.861378 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:11:56.087666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:56.087639 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:11:56.088222 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:56.088205 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:11:56.090653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:11:56.090633 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 08:12:01.402115 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:01.402070 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:02.569984 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:02.569953 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:03.569850 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:03.569814 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:12.070220 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:12.070187 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:12.169864 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:12.169826 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:38.716342 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:38.716310 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:39.505491 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:39.505458 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:42.300117 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:42.300081 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:42.354619 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:42.354589 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:43.893713 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:43.893681 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:44.027815 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:44.027789 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:44.231633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:44.231555 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:44.324610 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:44.324581 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:46.615826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:46.615796 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:47.002337 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.002259 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2thdq"] Jul 12 08:12:47.005796 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.005775 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.008296 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.008270 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 08:12:47.008437 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.008330 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hd9h8\"" Jul 12 08:12:47.008989 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.008970 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 08:12:47.018354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.018329 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2thdq"] Jul 12 08:12:47.035170 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.035137 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:47.083845 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.083813 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rkch\" (UniqueName: \"kubernetes.io/projected/08475880-938b-41dc-8943-cea247715e7c-kube-api-access-2rkch\") pod \"cert-manager-cainjector-559cdd8c87-2thdq\" (UID: \"08475880-938b-41dc-8943-cea247715e7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.083978 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.083893 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08475880-938b-41dc-8943-cea247715e7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2thdq\" (UID: \"08475880-938b-41dc-8943-cea247715e7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.184640 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.184609 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08475880-938b-41dc-8943-cea247715e7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2thdq\" (UID: \"08475880-938b-41dc-8943-cea247715e7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.184843 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.184657 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rkch\" (UniqueName: \"kubernetes.io/projected/08475880-938b-41dc-8943-cea247715e7c-kube-api-access-2rkch\") pod \"cert-manager-cainjector-559cdd8c87-2thdq\" (UID: \"08475880-938b-41dc-8943-cea247715e7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.192606 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.192583 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08475880-938b-41dc-8943-cea247715e7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2thdq\" (UID: \"08475880-938b-41dc-8943-cea247715e7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.192757 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.192723 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rkch\" (UniqueName: \"kubernetes.io/projected/08475880-938b-41dc-8943-cea247715e7c-kube-api-access-2rkch\") pod \"cert-manager-cainjector-559cdd8c87-2thdq\" (UID: \"08475880-938b-41dc-8943-cea247715e7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.316438 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.316408 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" Jul 12 08:12:47.435610 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.435576 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2thdq"] Jul 12 08:12:47.438659 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:12:47.438633 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08475880_938b_41dc_8943_cea247715e7c.slice/crio-22c37b54f444f37ffab6a383c3fbea1f53e319352edb338255c4014420b583e3 WatchSource:0}: Error finding container 22c37b54f444f37ffab6a383c3fbea1f53e319352edb338255c4014420b583e3: Status 404 returned error can't find the container with id 22c37b54f444f37ffab6a383c3fbea1f53e319352edb338255c4014420b583e3 Jul 12 08:12:47.440939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.440922 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:12:47.584268 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:47.584185 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" event={"ID":"08475880-938b-41dc-8943-cea247715e7c","Type":"ContainerStarted","Data":"22c37b54f444f37ffab6a383c3fbea1f53e319352edb338255c4014420b583e3"} Jul 12 08:12:49.096198 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:49.096112 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:49.197351 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:49.197311 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:51.600222 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:51.600183 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" event={"ID":"08475880-938b-41dc-8943-cea247715e7c","Type":"ContainerStarted","Data":"a6b40a076978eb1b37a49610f887f797ecec9c0e03f20a88ea2b46c6fee82fbd"} Jul 12 08:12:51.614620 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:51.614570 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2thdq" podStartSLOduration=2.500709529 podStartE2EDuration="5.614555514s" totalImagesPullingTime="3.113845985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:12:51.613214775 +0000 UTC m=+356.058699906" watchObservedRunningTime="2026-07-12 08:12:51.614555514 +0000 UTC m=+356.060040644" Jul 12 08:12:54.224417 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:54.224372 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:56.779899 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:56.779859 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:12:56.880784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:12:56.880757 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:02.571868 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:02.571834 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:02.672394 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:02.672361 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:04.305934 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:04.305905 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:17.049337 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:17.049299 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:17.120686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:17.120649 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:20.523621 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:20.523587 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:20.616651 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:20.616619 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.136309 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.136276 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.335072 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.335039 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.439163 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.439080 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.458191 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.458160 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8"] Jul 12 08:13:21.461088 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.461073 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.463471 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.463449 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mxs99\"" Jul 12 08:13:21.463594 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.463479 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 08:13:21.463594 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.463491 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 08:13:21.463594 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.463504 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 08:13:21.463594 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.463577 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 08:13:21.477092 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.477070 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8"] Jul 12 08:13:21.633040 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.633008 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.635851 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.635830 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-webhook-cert\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.635937 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.635862 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.635937 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.635887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-tmp\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.636040 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.635965 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-manifests\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.636040 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.636005 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68v7l\" (UniqueName: \"kubernetes.io/projected/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-kube-api-access-68v7l\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.735261 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.735171 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.737201 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.737180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-manifests\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.737260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.737212 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68v7l\" (UniqueName: \"kubernetes.io/projected/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-kube-api-access-68v7l\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.737366 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.737348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-webhook-cert\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.737424 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.737392 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.737474 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.737436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-tmp\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.737527 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.737514 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-manifests\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.739719 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.739693 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-tmp\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.740826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.740806 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-webhook-cert\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.740902 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.740814 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.750143 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.750121 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68v7l\" (UniqueName: \"kubernetes.io/projected/a2373797-ca9d-491c-ae97-cc4fdaa0f51a-kube-api-access-68v7l\") pod \"opendatahub-operator-controller-manager-595c444865-rwxr8\" (UID: \"a2373797-ca9d-491c-ae97-cc4fdaa0f51a\") " pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.770589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.770564 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:21.834927 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.834864 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.908693 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.908656 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8"] Jul 12 08:13:21.911753 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.911701 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:21.912142 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:13:21.912123 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2373797_ca9d_491c_ae97_cc4fdaa0f51a.slice/crio-30eec15f36820831fb1f5f9cb43d751192476eb5cb46fea34d694e20f1cf15ce WatchSource:0}: Error finding container 30eec15f36820831fb1f5f9cb43d751192476eb5cb46fea34d694e20f1cf15ce: Status 404 returned error can't find the container with id 30eec15f36820831fb1f5f9cb43d751192476eb5cb46fea34d694e20f1cf15ce Jul 12 08:13:21.978509 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:21.978477 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:22.072843 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:22.072818 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:22.173230 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:22.173199 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:22.699414 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:22.699379 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" event={"ID":"a2373797-ca9d-491c-ae97-cc4fdaa0f51a","Type":"ContainerStarted","Data":"30eec15f36820831fb1f5f9cb43d751192476eb5cb46fea34d694e20f1cf15ce"} Jul 12 08:13:24.539503 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:24.539457 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:24.640251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:24.640218 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:25.712610 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:25.712575 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" event={"ID":"a2373797-ca9d-491c-ae97-cc4fdaa0f51a","Type":"ContainerStarted","Data":"3ab610f41c986fde7fe888eb52267cc94fee372c307bbfac27a0402e906e062e"} Jul 12 08:13:26.717686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:26.717649 2558 generic.go:356] "Generic (PLEG): container finished" podID="a2373797-ca9d-491c-ae97-cc4fdaa0f51a" containerID="3ab610f41c986fde7fe888eb52267cc94fee372c307bbfac27a0402e906e062e" exitCode=0 Jul 12 08:13:26.718131 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:26.717731 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" event={"ID":"a2373797-ca9d-491c-ae97-cc4fdaa0f51a","Type":"ContainerDied","Data":"3ab610f41c986fde7fe888eb52267cc94fee372c307bbfac27a0402e906e062e"} Jul 12 08:13:27.722468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:27.722431 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" event={"ID":"a2373797-ca9d-491c-ae97-cc4fdaa0f51a","Type":"ContainerStarted","Data":"b3d6899336c215c171c447baf5bf93baf474c7e0384fe168f8f0de6b12ec3f47"} Jul 12 08:13:27.722854 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:27.722539 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:27.748249 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:27.748199 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" podStartSLOduration=2.297345604 podStartE2EDuration="6.74818436s" totalImagesPullingTime="3.408501677s" totalInitContainerRuntime="1.042337079s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:13:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:13:27.746312715 +0000 UTC m=+392.191797845" watchObservedRunningTime="2026-07-12 08:13:27.74818436 +0000 UTC m=+392.193669492" Jul 12 08:13:33.544503 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:33.544471 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:33.642408 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:33.642372 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:38.728216 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:38.728184 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-595c444865-rwxr8" Jul 12 08:13:40.333067 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.333035 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:40.437864 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.437830 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:40.779790 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.779698 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:40.906842 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.906807 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7"] Jul 12 08:13:40.919821 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.919795 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:40.921876 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.921850 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7"] Jul 12 08:13:40.922020 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.921929 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 08:13:40.922020 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.921929 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 08:13:40.922208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.922193 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 08:13:40.922266 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.922227 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 08:13:40.923080 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.923064 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bh65g\"" Jul 12 08:13:40.972943 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.972905 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwljl\" (UniqueName: \"kubernetes.io/projected/0c221d97-63d9-4eae-b30f-bb2b5bf91714-kube-api-access-mwljl\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:40.972943 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.972948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tls-certs\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:40.973129 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:40.972964 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tmp\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.048830 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.048727 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fgbrl"] Jul 12 08:13:41.052381 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.052364 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.054767 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.054722 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tp4ps\"" Jul 12 08:13:41.055134 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.055120 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 12 08:13:41.061057 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.061030 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fgbrl"] Jul 12 08:13:41.073529 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.073503 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tls-certs\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.073665 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.073540 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tmp\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.073665 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:13:41.073653 2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 12 08:13:41.073802 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:13:41.073730 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tls-certs podName:0c221d97-63d9-4eae-b30f-bb2b5bf91714 nodeName:}" failed. No retries permitted until 2026-07-12 08:13:41.573708411 +0000 UTC m=+406.019193521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tls-certs") pod "kube-auth-proxy-65cfff4589-kcbv7" (UID: "0c221d97-63d9-4eae-b30f-bb2b5bf91714") : secret "kube-auth-proxy-tls" not found Jul 12 08:13:41.073891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.073869 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f398045d-0d89-4774-bd23-890d1892bd7f-cert\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.074417 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.074380 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q96ln\" (UniqueName: \"kubernetes.io/projected/f398045d-0d89-4774-bd23-890d1892bd7f-kube-api-access-q96ln\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.074510 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.074483 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwljl\" (UniqueName: \"kubernetes.io/projected/0c221d97-63d9-4eae-b30f-bb2b5bf91714-kube-api-access-mwljl\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.074975 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.074951 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:41.076101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.076079 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tmp\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.091827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.091807 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwljl\" (UniqueName: \"kubernetes.io/projected/0c221d97-63d9-4eae-b30f-bb2b5bf91714-kube-api-access-mwljl\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.174843 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.174809 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f398045d-0d89-4774-bd23-890d1892bd7f-cert\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.175030 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.174851 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q96ln\" (UniqueName: \"kubernetes.io/projected/f398045d-0d89-4774-bd23-890d1892bd7f-kube-api-access-q96ln\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.175030 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:13:41.174965 2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 12 08:13:41.175139 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:13:41.175035 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f398045d-0d89-4774-bd23-890d1892bd7f-cert podName:f398045d-0d89-4774-bd23-890d1892bd7f nodeName:}" failed. No retries permitted until 2026-07-12 08:13:41.67501589 +0000 UTC m=+406.120501001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f398045d-0d89-4774-bd23-890d1892bd7f-cert") pod "odh-model-controller-7998c67dc4-fgbrl" (UID: "f398045d-0d89-4774-bd23-890d1892bd7f") : secret "odh-model-controller-webhook-cert" not found Jul 12 08:13:41.176098 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.176073 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:41.182837 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.182816 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q96ln\" (UniqueName: \"kubernetes.io/projected/f398045d-0d89-4774-bd23-890d1892bd7f-kube-api-access-q96ln\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.275239 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.275203 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:41.381841 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.381806 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:41.489101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.489070 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:41.578317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.578283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tls-certs\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.580668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.580639 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c221d97-63d9-4eae-b30f-bb2b5bf91714-tls-certs\") pod \"kube-auth-proxy-65cfff4589-kcbv7\" (UID: \"0c221d97-63d9-4eae-b30f-bb2b5bf91714\") " pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.678965 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.678865 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f398045d-0d89-4774-bd23-890d1892bd7f-cert\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.681306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.681274 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f398045d-0d89-4774-bd23-890d1892bd7f-cert\") pod \"odh-model-controller-7998c67dc4-fgbrl\" (UID: \"f398045d-0d89-4774-bd23-890d1892bd7f\") " pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.721269 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.721241 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g5w7p"] Jul 12 08:13:41.724620 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.724600 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:41.727244 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.727217 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 12 08:13:41.736269 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.736249 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g5w7p"] Jul 12 08:13:41.741438 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.741415 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:41.779268 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.779240 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnkh8\" (UniqueName: \"kubernetes.io/projected/ed416917-3fb6-43a5-992e-2fb8367ad43a-kube-api-access-tnkh8\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:41.779430 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.779290 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed416917-3fb6-43a5-992e-2fb8367ad43a-tls-certs\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:41.831427 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.831395 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" Jul 12 08:13:41.880460 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.880080 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnkh8\" (UniqueName: \"kubernetes.io/projected/ed416917-3fb6-43a5-992e-2fb8367ad43a-kube-api-access-tnkh8\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:41.880460 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.880141 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed416917-3fb6-43a5-992e-2fb8367ad43a-tls-certs\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:41.880460 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:13:41.880260 2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 12 08:13:41.880460 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:13:41.880332 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed416917-3fb6-43a5-992e-2fb8367ad43a-tls-certs podName:ed416917-3fb6-43a5-992e-2fb8367ad43a nodeName:}" failed. No retries permitted until 2026-07-12 08:13:42.380312398 +0000 UTC m=+406.825797519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ed416917-3fb6-43a5-992e-2fb8367ad43a-tls-certs") pod "model-serving-api-54d89bd5fc-g5w7p" (UID: "ed416917-3fb6-43a5-992e-2fb8367ad43a") : secret "model-serving-api-tls" not found Jul 12 08:13:41.888250 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.888218 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnkh8\" (UniqueName: \"kubernetes.io/projected/ed416917-3fb6-43a5-992e-2fb8367ad43a-kube-api-access-tnkh8\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:41.963259 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.963182 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:41.963892 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:41.963804 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7"] Jul 12 08:13:41.966720 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:13:41.966696 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c221d97_63d9_4eae_b30f_bb2b5bf91714.slice/crio-61f21207283bf39755745204a6f27c8982d22dce345357c8a24d7dc1fc151111 WatchSource:0}: Error finding container 61f21207283bf39755745204a6f27c8982d22dce345357c8a24d7dc1fc151111: Status 404 returned error can't find the container with id 61f21207283bf39755745204a6f27c8982d22dce345357c8a24d7dc1fc151111 Jul 12 08:13:42.053373 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.053334 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:42.095393 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.095362 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fgbrl"] Jul 12 08:13:42.099563 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:13:42.099521 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf398045d_0d89_4774_bd23_890d1892bd7f.slice/crio-b4759cf896143495916d3201f5e730dc6c5caef3be9a1c03e32d80f0e978a8aa WatchSource:0}: Error finding container b4759cf896143495916d3201f5e730dc6c5caef3be9a1c03e32d80f0e978a8aa: Status 404 returned error can't find the container with id b4759cf896143495916d3201f5e730dc6c5caef3be9a1c03e32d80f0e978a8aa Jul 12 08:13:42.145289 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.145248 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:42.204679 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.204650 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:42.383340 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.383306 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed416917-3fb6-43a5-992e-2fb8367ad43a-tls-certs\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:42.386078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.386046 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed416917-3fb6-43a5-992e-2fb8367ad43a-tls-certs\") pod \"model-serving-api-54d89bd5fc-g5w7p\" (UID: \"ed416917-3fb6-43a5-992e-2fb8367ad43a\") " pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:42.634370 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.634278 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:42.776211 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.776170 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" event={"ID":"f398045d-0d89-4774-bd23-890d1892bd7f","Type":"ContainerStarted","Data":"b4759cf896143495916d3201f5e730dc6c5caef3be9a1c03e32d80f0e978a8aa"} Jul 12 08:13:42.777255 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.777227 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g5w7p"] Jul 12 08:13:42.777574 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.777554 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" event={"ID":"0c221d97-63d9-4eae-b30f-bb2b5bf91714","Type":"ContainerStarted","Data":"61f21207283bf39755745204a6f27c8982d22dce345357c8a24d7dc1fc151111"} Jul 12 08:13:42.779830 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:13:42.779806 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded416917_3fb6_43a5_992e_2fb8367ad43a.slice/crio-18b35f16a87ab3c33a2eda58d86d42eb86bd87e4c02c4b8e19345c3003630075 WatchSource:0}: Error finding container 18b35f16a87ab3c33a2eda58d86d42eb86bd87e4c02c4b8e19345c3003630075: Status 404 returned error can't find the container with id 18b35f16a87ab3c33a2eda58d86d42eb86bd87e4c02c4b8e19345c3003630075 Jul 12 08:13:42.974661 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:42.974573 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:43.405262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:43.405182 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:43.456074 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:43.456032 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:43.783263 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:43.783155 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" event={"ID":"ed416917-3fb6-43a5-992e-2fb8367ad43a","Type":"ContainerStarted","Data":"18b35f16a87ab3c33a2eda58d86d42eb86bd87e4c02c4b8e19345c3003630075"} Jul 12 08:13:45.482921 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:45.482888 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:45.546436 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:45.546404 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:46.029573 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.029528 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt"] Jul 12 08:13:46.032891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.032871 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.035531 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.035508 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ckll2\"" Jul 12 08:13:46.035774 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.035757 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 08:13:46.046701 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.046676 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt"] Jul 12 08:13:46.069399 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.069364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:46.114087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114046 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114106 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27bsk\" (UniqueName: \"kubernetes.io/projected/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-kube-api-access-27bsk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114148 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114236 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114269 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114297 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114325 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114360 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.114481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.114388 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.214052 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.213966 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:46.214923 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.214904 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.214981 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.214934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.214981 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.214954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.214979 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215000 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215052 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215252 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27bsk\" (UniqueName: \"kubernetes.io/projected/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-kube-api-access-27bsk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215252 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215151 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215399 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215358 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215446 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215566 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215538 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.215566 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.215559 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.216038 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.216019 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.217379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.217358 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.217649 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.217631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.223404 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.223377 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27bsk\" (UniqueName: \"kubernetes.io/projected/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-kube-api-access-27bsk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.224653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.224631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8c43c1-5263-4086-baf0-c7dcfb1593c3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v2qnt\" (UID: \"0b8c43c1-5263-4086-baf0-c7dcfb1593c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.343208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.343167 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:46.488324 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.488226 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt"] Jul 12 08:13:46.491307 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:13:46.491281 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8c43c1_5263_4086_baf0_c7dcfb1593c3.slice/crio-deb22a3b2e860213d18ffe508ce9a5e5384241c6b04f9340b2c85dd63110aa8c WatchSource:0}: Error finding container deb22a3b2e860213d18ffe508ce9a5e5384241c6b04f9340b2c85dd63110aa8c: Status 404 returned error can't find the container with id deb22a3b2e860213d18ffe508ce9a5e5384241c6b04f9340b2c85dd63110aa8c Jul 12 08:13:46.758339 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.758243 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:46.797266 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.797230 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" event={"ID":"f398045d-0d89-4774-bd23-890d1892bd7f","Type":"ContainerStarted","Data":"94069172cf1506052e657f1cc6dcb2102bff616972a4b1c29d6c8f199c614fd2"} Jul 12 08:13:46.797420 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.797368 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:46.798470 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.798444 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" event={"ID":"0b8c43c1-5263-4086-baf0-c7dcfb1593c3","Type":"ContainerStarted","Data":"deb22a3b2e860213d18ffe508ce9a5e5384241c6b04f9340b2c85dd63110aa8c"} Jul 12 08:13:46.799780 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.799703 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" event={"ID":"0c221d97-63d9-4eae-b30f-bb2b5bf91714","Type":"ContainerStarted","Data":"2ec10d16701b4948b5224d9d76e002beed7672bcb924aaaa8c7b30cc73814d2a"} Jul 12 08:13:46.801159 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.801138 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" event={"ID":"ed416917-3fb6-43a5-992e-2fb8367ad43a","Type":"ContainerStarted","Data":"a14e3b3980af1a546f25c23d132399ee7f456be3b8a5e0cf3445e8784361332f"} Jul 12 08:13:46.801300 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.801277 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:13:46.811791 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.811736 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" podStartSLOduration=2.021903484 podStartE2EDuration="5.811722569s" totalImagesPullingTime="3.789819085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:13:46.81062263 +0000 UTC m=+411.256107757" watchObservedRunningTime="2026-07-12 08:13:46.811722569 +0000 UTC m=+411.257207698" Jul 12 08:13:46.825944 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.825899 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65cfff4589-kcbv7" podStartSLOduration=2.916108275 podStartE2EDuration="6.825885331s" totalImagesPullingTime="3.909777056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:13:46.82443098 +0000 UTC m=+411.269916132" watchObservedRunningTime="2026-07-12 08:13:46.825885331 +0000 UTC m=+411.271370465" Jul 12 08:13:46.840823 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:46.840770 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" podStartSLOduration=2.74018189 podStartE2EDuration="5.8407352s" totalImagesPullingTime="3.10055331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:13:46.840236683 +0000 UTC m=+411.285721813" watchObservedRunningTime="2026-07-12 08:13:46.8407352 +0000 UTC m=+411.286220332" Jul 12 08:13:49.127816 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.127781 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:13:49.128114 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.127879 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:13:49.128114 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.127913 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:13:49.717825 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.717789 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg"] Jul 12 08:13:49.722120 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.722096 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.726440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.725930 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 08:13:49.726440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.725948 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 08:13:49.726440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.726221 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 08:13:49.726440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.726332 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 08:13:49.726440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.726387 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2qg5q\"" Jul 12 08:13:49.726801 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.726605 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:13:49.733663 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.733639 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg"] Jul 12 08:13:49.744484 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.744461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/752320ec-efe9-40c9-82c2-6d1f3a965a61-manager-config\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.744598 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.744489 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p7w6\" (UniqueName: \"kubernetes.io/projected/752320ec-efe9-40c9-82c2-6d1f3a965a61-kube-api-access-8p7w6\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.744598 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.744521 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/752320ec-efe9-40c9-82c2-6d1f3a965a61-cert\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.744665 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.744627 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/752320ec-efe9-40c9-82c2-6d1f3a965a61-metrics-cert\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.813435 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.813400 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" event={"ID":"0b8c43c1-5263-4086-baf0-c7dcfb1593c3","Type":"ContainerStarted","Data":"572dda9c64332131dd52f34354fc37ea33defe0121da81d83c12f6baab491be1"} Jul 12 08:13:49.835826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.835770 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" podStartSLOduration=1.201698538 podStartE2EDuration="3.835738051s" totalImagesPullingTime="2.634039513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:13:49.834425281 +0000 UTC m=+414.279910450" watchObservedRunningTime="2026-07-12 08:13:49.835738051 +0000 UTC m=+414.281223182" Jul 12 08:13:49.845667 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.845637 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/752320ec-efe9-40c9-82c2-6d1f3a965a61-metrics-cert\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.845944 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.845920 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/752320ec-efe9-40c9-82c2-6d1f3a965a61-manager-config\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.846045 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.845958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p7w6\" (UniqueName: \"kubernetes.io/projected/752320ec-efe9-40c9-82c2-6d1f3a965a61-kube-api-access-8p7w6\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.846045 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.846004 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/752320ec-efe9-40c9-82c2-6d1f3a965a61-cert\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.846586 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.846561 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/752320ec-efe9-40c9-82c2-6d1f3a965a61-manager-config\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.848107 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.848086 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/752320ec-efe9-40c9-82c2-6d1f3a965a61-metrics-cert\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.848184 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.848165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/752320ec-efe9-40c9-82c2-6d1f3a965a61-cert\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:49.853632 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:49.853610 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p7w6\" (UniqueName: \"kubernetes.io/projected/752320ec-efe9-40c9-82c2-6d1f3a965a61-kube-api-access-8p7w6\") pod \"lws-controller-manager-59795878b4-8v8tg\" (UID: \"752320ec-efe9-40c9-82c2-6d1f3a965a61\") " pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:50.034442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.034345 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:50.176732 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.176703 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg"] Jul 12 08:13:50.178574 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:13:50.178548 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752320ec_efe9_40c9_82c2_6d1f3a965a61.slice/crio-a963756a27c02ac11a149a8766c03eeed984100057960d4c437cd317920bb8bf WatchSource:0}: Error finding container a963756a27c02ac11a149a8766c03eeed984100057960d4c437cd317920bb8bf: Status 404 returned error can't find the container with id a963756a27c02ac11a149a8766c03eeed984100057960d4c437cd317920bb8bf Jul 12 08:13:50.343817 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.343782 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:50.348876 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.348853 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:50.818609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.818571 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" event={"ID":"752320ec-efe9-40c9-82c2-6d1f3a965a61","Type":"ContainerStarted","Data":"a963756a27c02ac11a149a8766c03eeed984100057960d4c437cd317920bb8bf"} Jul 12 08:13:50.818790 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.818764 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:50.819642 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:50.819624 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v2qnt" Jul 12 08:13:51.064403 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:51.064338 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:51.211032 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:51.209525 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:51.432889 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:51.432856 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:51.576329 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:51.576291 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:52.400380 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:52.400348 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:52.502347 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:52.502312 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:52.602275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:52.602244 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:13:52.828615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:52.828580 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" event={"ID":"752320ec-efe9-40c9-82c2-6d1f3a965a61","Type":"ContainerStarted","Data":"07a625a4d431fea77586d7da02fb8163f8ec76c30bfe3bb97b67b662bb11e6ac"} Jul 12 08:13:52.828821 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:52.828784 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:13:52.851369 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:52.851307 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" podStartSLOduration=1.407443175 podStartE2EDuration="3.851290082s" totalImagesPullingTime="2.443846907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:13:52.850037708 +0000 UTC m=+417.295522857" watchObservedRunningTime="2026-07-12 08:13:52.851290082 +0000 UTC m=+417.296775209" Jul 12 08:13:57.806530 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:57.806490 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fgbrl" Jul 12 08:13:57.808549 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:13:57.808526 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-g5w7p" Jul 12 08:14:03.835005 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:14:03.834975 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59795878b4-8v8tg" Jul 12 08:14:41.866666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:14:41.866637 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:14:41.963547 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:14:41.963515 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:14:42.063448 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:14:42.063414 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:00.326685 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:00.326648 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:00.472637 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:00.472603 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:00.518795 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:00.518764 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:01.077817 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:01.077780 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:01.178083 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:01.178048 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:10.796704 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:10.796671 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:11.442398 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:11.442369 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:11.543558 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:11.543526 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:11.643294 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:11.643261 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:11.696961 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:11.696883 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:14.959845 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:14.959812 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:17.365330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:17.365299 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:17.659828 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:17.659728 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:17.963087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:17.963009 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:21.162933 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:21.162903 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:22.598449 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:22.598416 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:24.605211 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:24.605176 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:26.412634 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:26.412603 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:27.368095 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:27.368057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:27.468537 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:27.468508 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:31.179098 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:31.179062 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:31.280079 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:31.280048 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:33.972859 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:33.972822 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:34.444345 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:34.444316 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:35.367881 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:35.367845 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:35.466120 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:35.466081 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:35.789884 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:35.789796 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:35.890981 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:35.890946 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:39.753048 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:39.753016 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:39.853292 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:39.853261 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:46.318438 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:46.318401 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:46.422033 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:46.421972 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:49.134099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:49.134000 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:49.255094 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:49.255061 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:53.937995 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:53.937959 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:54.039549 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:54.039517 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:57.609938 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:57.609902 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:57.749783 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:57.749728 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:15:59.562833 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:15:59.562801 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:04.104607 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:04.104576 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:04.200347 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:04.200316 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:07.298370 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:07.298339 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:07.397724 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:07.397695 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:10.056876 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:10.056848 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:11.894451 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:11.894416 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:11.962319 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:11.962288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:12.100274 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.100242 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:12.171788 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.171691 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:12.968316 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.968284 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c"] Jul 12 08:16:12.971454 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.971432 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:12.973591 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.973572 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-m9l76\"" Jul 12 08:16:12.982961 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.982940 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c"] Jul 12 08:16:12.996280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:12.996255 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:13.075419 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075388 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075434 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075456 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075527 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075865 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075633 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075865 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075692 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.075865 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.075734 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gcs\" (UniqueName: \"kubernetes.io/projected/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-kube-api-access-v9gcs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.107707 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.107680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:13.176809 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.176766 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.176966 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.176827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.176966 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.176851 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.176966 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.176879 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.176966 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.176901 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177157 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177027 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177157 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177157 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177112 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177157 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gcs\" (UniqueName: \"kubernetes.io/projected/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-kube-api-access-v9gcs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177264 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177289 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177493 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177411 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177552 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177521 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.177799 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.177780 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.179221 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.179194 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.179412 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.179397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.184716 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.184689 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.184915 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.184898 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gcs\" (UniqueName: \"kubernetes.io/projected/decfcf79-5029-46b2-8a3b-9d70fbfc5ee3-kube-api-access-v9gcs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz94c\" (UID: \"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.283869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.283782 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:13.413343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.413312 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c"] Jul 12 08:16:13.415800 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:16:13.415774 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecfcf79_5029_46b2_8a3b_9d70fbfc5ee3.slice/crio-94176aeeb33dde7630b9c487aae0dd1f894ca30ac4be29eaea4065b1b63a3c05 WatchSource:0}: Error finding container 94176aeeb33dde7630b9c487aae0dd1f894ca30ac4be29eaea4065b1b63a3c05: Status 404 returned error can't find the container with id 94176aeeb33dde7630b9c487aae0dd1f894ca30ac4be29eaea4065b1b63a3c05 Jul 12 08:16:13.417850 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.417821 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:16:13.417913 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.417880 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:16:13.417952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:13.417914 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:16:14.359090 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:14.359056 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" event={"ID":"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3","Type":"ContainerStarted","Data":"ff4872fa6199e95388b209a89cffe012859cc53a6cb3b2ad377de1af87aa68fb"} Jul 12 08:16:14.359577 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:14.359113 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" event={"ID":"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3","Type":"ContainerStarted","Data":"94176aeeb33dde7630b9c487aae0dd1f894ca30ac4be29eaea4065b1b63a3c05"} Jul 12 08:16:14.383240 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:14.383185 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" podStartSLOduration=2.383169117 podStartE2EDuration="2.383169117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:16:14.38267741 +0000 UTC m=+558.828162542" watchObservedRunningTime="2026-07-12 08:16:14.383169117 +0000 UTC m=+558.828654247" Jul 12 08:16:15.284785 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:15.284728 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:15.289070 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:15.289049 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:15.362711 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:15.362675 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:15.363659 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:15.363639 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:16:17.114394 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:17.114359 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:17.182995 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:17.182967 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:31.380910 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:31.380873 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:32.098498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.098465 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:32.199229 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.199197 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:32.251483 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.251449 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:32.901945 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.901911 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6p9sv"] Jul 12 08:16:32.905025 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.905008 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:32.908431 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.908411 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 08:16:32.908550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.908412 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5l5x7\"" Jul 12 08:16:32.908550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.908414 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 08:16:32.914440 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.914416 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6p9sv"] Jul 12 08:16:32.942458 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:32.942426 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:33.042539 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.042505 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd8f8\" (UniqueName: \"kubernetes.io/projected/9fa5c4d6-a297-4345-9060-deefa6e1fb7d-kube-api-access-bd8f8\") pod \"authorino-58cd955675-6p9sv\" (UID: \"9fa5c4d6-a297-4345-9060-deefa6e1fb7d\") " pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:33.043300 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.043273 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:33.143405 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.143365 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd8f8\" (UniqueName: \"kubernetes.io/projected/9fa5c4d6-a297-4345-9060-deefa6e1fb7d-kube-api-access-bd8f8\") pod \"authorino-58cd955675-6p9sv\" (UID: \"9fa5c4d6-a297-4345-9060-deefa6e1fb7d\") " pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:33.152504 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.152427 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd8f8\" (UniqueName: \"kubernetes.io/projected/9fa5c4d6-a297-4345-9060-deefa6e1fb7d-kube-api-access-bd8f8\") pod \"authorino-58cd955675-6p9sv\" (UID: \"9fa5c4d6-a297-4345-9060-deefa6e1fb7d\") " pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:33.215376 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.215338 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:33.344311 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.344283 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6p9sv"] Jul 12 08:16:33.346945 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:16:33.346924 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fa5c4d6_a297_4345_9060_deefa6e1fb7d.slice/crio-ccadae4af72de2c72555fd7893e8be087cf55657a9d8f2ca5ee373911f05998c WatchSource:0}: Error finding container ccadae4af72de2c72555fd7893e8be087cf55657a9d8f2ca5ee373911f05998c: Status 404 returned error can't find the container with id ccadae4af72de2c72555fd7893e8be087cf55657a9d8f2ca5ee373911f05998c Jul 12 08:16:33.433853 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:33.433772 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6p9sv" event={"ID":"9fa5c4d6-a297-4345-9060-deefa6e1fb7d","Type":"ContainerStarted","Data":"ccadae4af72de2c72555fd7893e8be087cf55657a9d8f2ca5ee373911f05998c"} Jul 12 08:16:37.453233 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:37.453193 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6p9sv" event={"ID":"9fa5c4d6-a297-4345-9060-deefa6e1fb7d","Type":"ContainerStarted","Data":"0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e"} Jul 12 08:16:37.466902 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:37.466853 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6p9sv" podStartSLOduration=2.29143244 podStartE2EDuration="5.466838604s" totalImagesPullingTime="3.175406164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:16:37.466055616 +0000 UTC m=+581.911540747" watchObservedRunningTime="2026-07-12 08:16:37.466838604 +0000 UTC m=+581.912323734" Jul 12 08:16:37.773805 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:37.773712 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6p9sv"] Jul 12 08:16:39.461596 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:39.461559 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6p9sv" podUID="9fa5c4d6-a297-4345-9060-deefa6e1fb7d" containerName="authorino" containerID="cri-o://0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e" gracePeriod=30 Jul 12 08:16:39.697128 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:39.697101 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:39.800248 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:39.800152 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd8f8\" (UniqueName: \"kubernetes.io/projected/9fa5c4d6-a297-4345-9060-deefa6e1fb7d-kube-api-access-bd8f8\") pod \"9fa5c4d6-a297-4345-9060-deefa6e1fb7d\" (UID: \"9fa5c4d6-a297-4345-9060-deefa6e1fb7d\") " Jul 12 08:16:39.802329 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:39.802295 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa5c4d6-a297-4345-9060-deefa6e1fb7d-kube-api-access-bd8f8" (OuterVolumeSpecName: "kube-api-access-bd8f8") pod "9fa5c4d6-a297-4345-9060-deefa6e1fb7d" (UID: "9fa5c4d6-a297-4345-9060-deefa6e1fb7d"). InnerVolumeSpecName "kube-api-access-bd8f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:16:39.900873 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:39.900825 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd8f8\" (UniqueName: \"kubernetes.io/projected/9fa5c4d6-a297-4345-9060-deefa6e1fb7d-kube-api-access-bd8f8\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:16:40.466616 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.466581 2558 generic.go:356] "Generic (PLEG): container finished" podID="9fa5c4d6-a297-4345-9060-deefa6e1fb7d" containerID="0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e" exitCode=0 Jul 12 08:16:40.467052 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.466623 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6p9sv" Jul 12 08:16:40.467052 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.466633 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6p9sv" event={"ID":"9fa5c4d6-a297-4345-9060-deefa6e1fb7d","Type":"ContainerDied","Data":"0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e"} Jul 12 08:16:40.467052 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.466663 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6p9sv" event={"ID":"9fa5c4d6-a297-4345-9060-deefa6e1fb7d","Type":"ContainerDied","Data":"ccadae4af72de2c72555fd7893e8be087cf55657a9d8f2ca5ee373911f05998c"} Jul 12 08:16:40.467052 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.466679 2558 scope.go:117] "RemoveContainer" containerID="0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e" Jul 12 08:16:40.474655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.474635 2558 scope.go:117] "RemoveContainer" containerID="0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e" Jul 12 08:16:40.474919 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:16:40.474898 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e\": container with ID starting with 0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e not found: ID does not exist" containerID="0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e" Jul 12 08:16:40.474978 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.474929 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e"} err="failed to get container status \"0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e\": rpc error: code = NotFound desc = could not find container \"0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e\": container with ID starting with 0542307122e1cf8e3d5c529d34a260fc04a06fa01f40114f6faa91be4776ae3e not found: ID does not exist" Jul 12 08:16:40.481339 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.481315 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6p9sv"] Jul 12 08:16:40.484951 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:40.484928 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6p9sv"] Jul 12 08:16:42.387151 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:42.387115 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fa5c4d6-a297-4345-9060-deefa6e1fb7d" path="/var/lib/kubelet/pods/9fa5c4d6-a297-4345-9060-deefa6e1fb7d/volumes" Jul 12 08:16:52.025567 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:52.025529 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:52.124885 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:52.124852 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:52.482416 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:52.482380 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:55.061593 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:55.061558 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:55.112448 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:55.112414 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:16:56.111022 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:56.110995 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:16:56.111460 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:16:56.111126 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:17:05.208014 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.207976 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:05.560678 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.560648 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vsmr7"] Jul 12 08:17:05.560985 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.560972 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa5c4d6-a297-4345-9060-deefa6e1fb7d" containerName="authorino" Jul 12 08:17:05.561034 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.560986 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa5c4d6-a297-4345-9060-deefa6e1fb7d" containerName="authorino" Jul 12 08:17:05.561068 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.561035 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa5c4d6-a297-4345-9060-deefa6e1fb7d" containerName="authorino" Jul 12 08:17:05.563684 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.563669 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:05.565680 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.565663 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 08:17:05.566464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.566445 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5l5x7\"" Jul 12 08:17:05.566560 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.566480 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 08:17:05.587063 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.587037 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt9xw\" (UniqueName: \"kubernetes.io/projected/42b58e27-70f4-4d1d-8b54-2c5db60270b0-kube-api-access-dt9xw\") pod \"authorino-5d4b67c446-vsmr7\" (UID: \"42b58e27-70f4-4d1d-8b54-2c5db60270b0\") " pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:05.590840 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.590816 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vsmr7"] Jul 12 08:17:05.688336 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.688302 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt9xw\" (UniqueName: \"kubernetes.io/projected/42b58e27-70f4-4d1d-8b54-2c5db60270b0-kube-api-access-dt9xw\") pod \"authorino-5d4b67c446-vsmr7\" (UID: \"42b58e27-70f4-4d1d-8b54-2c5db60270b0\") " pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:05.697426 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.697396 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt9xw\" (UniqueName: \"kubernetes.io/projected/42b58e27-70f4-4d1d-8b54-2c5db60270b0-kube-api-access-dt9xw\") pod \"authorino-5d4b67c446-vsmr7\" (UID: \"42b58e27-70f4-4d1d-8b54-2c5db60270b0\") " pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:05.714278 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.714250 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vsmr7"] Jul 12 08:17:05.714487 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.714475 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:05.747442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.747412 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57bf79ccd6-49pcv"] Jul 12 08:17:05.750628 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.750611 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:05.772284 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.772260 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57bf79ccd6-49pcv"] Jul 12 08:17:05.789035 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.789004 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-484ts\" (UniqueName: \"kubernetes.io/projected/0e2a6615-dab6-4bea-afcf-9ba07efd894d-kube-api-access-484ts\") pod \"authorino-57bf79ccd6-49pcv\" (UID: \"0e2a6615-dab6-4bea-afcf-9ba07efd894d\") " pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:05.847724 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.847643 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vsmr7"] Jul 12 08:17:05.890242 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.890213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-484ts\" (UniqueName: \"kubernetes.io/projected/0e2a6615-dab6-4bea-afcf-9ba07efd894d-kube-api-access-484ts\") pod \"authorino-57bf79ccd6-49pcv\" (UID: \"0e2a6615-dab6-4bea-afcf-9ba07efd894d\") " pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:05.899542 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.899516 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-484ts\" (UniqueName: \"kubernetes.io/projected/0e2a6615-dab6-4bea-afcf-9ba07efd894d-kube-api-access-484ts\") pod \"authorino-57bf79ccd6-49pcv\" (UID: \"0e2a6615-dab6-4bea-afcf-9ba07efd894d\") " pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:05.985344 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.985308 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57bf79ccd6-49pcv"] Jul 12 08:17:05.985548 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:05.985536 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:06.007277 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.007245 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dbd48c4c8-jvqw2"] Jul 12 08:17:06.010479 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.010461 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.012890 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.012866 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 12 08:17:06.015972 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.015951 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dbd48c4c8-jvqw2"] Jul 12 08:17:06.091242 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.091213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88cf8b34-f4bf-422d-bce6-1defb9bb2170-tls-cert\") pod \"authorino-6dbd48c4c8-jvqw2\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.091413 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.091278 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr66h\" (UniqueName: \"kubernetes.io/projected/88cf8b34-f4bf-422d-bce6-1defb9bb2170-kube-api-access-vr66h\") pod \"authorino-6dbd48c4c8-jvqw2\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.114568 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.114484 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57bf79ccd6-49pcv"] Jul 12 08:17:06.118289 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:17:06.118261 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e2a6615_dab6_4bea_afcf_9ba07efd894d.slice/crio-3d26754c6761b7104c8837a9c93fbd771f3d55899ce1221bc779c9195a96b8a8 WatchSource:0}: Error finding container 3d26754c6761b7104c8837a9c93fbd771f3d55899ce1221bc779c9195a96b8a8: Status 404 returned error can't find the container with id 3d26754c6761b7104c8837a9c93fbd771f3d55899ce1221bc779c9195a96b8a8 Jul 12 08:17:06.192732 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.192696 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88cf8b34-f4bf-422d-bce6-1defb9bb2170-tls-cert\") pod \"authorino-6dbd48c4c8-jvqw2\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.192869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.192813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr66h\" (UniqueName: \"kubernetes.io/projected/88cf8b34-f4bf-422d-bce6-1defb9bb2170-kube-api-access-vr66h\") pod \"authorino-6dbd48c4c8-jvqw2\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.195023 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.194999 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88cf8b34-f4bf-422d-bce6-1defb9bb2170-tls-cert\") pod \"authorino-6dbd48c4c8-jvqw2\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.201418 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.201398 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr66h\" (UniqueName: \"kubernetes.io/projected/88cf8b34-f4bf-422d-bce6-1defb9bb2170-kube-api-access-vr66h\") pod \"authorino-6dbd48c4c8-jvqw2\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.254892 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.254860 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:06.321444 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.321402 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:17:06.444371 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.444347 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dbd48c4c8-jvqw2"] Jul 12 08:17:06.446818 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:17:06.446785 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88cf8b34_f4bf_422d_bce6_1defb9bb2170.slice/crio-f79cd2d75a302c8c33449d4ef23d168ce7e5b47d3114b80acd52d0ee19f03443 WatchSource:0}: Error finding container f79cd2d75a302c8c33449d4ef23d168ce7e5b47d3114b80acd52d0ee19f03443: Status 404 returned error can't find the container with id f79cd2d75a302c8c33449d4ef23d168ce7e5b47d3114b80acd52d0ee19f03443 Jul 12 08:17:06.565144 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.565096 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" event={"ID":"42b58e27-70f4-4d1d-8b54-2c5db60270b0","Type":"ContainerStarted","Data":"fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53"} Jul 12 08:17:06.565144 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.565145 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" event={"ID":"42b58e27-70f4-4d1d-8b54-2c5db60270b0","Type":"ContainerStarted","Data":"91c7698175a71f70824fba32b118782a5cf3a0388d6f58f341927429b35bb75f"} Jul 12 08:17:06.565394 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.565217 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" podUID="42b58e27-70f4-4d1d-8b54-2c5db60270b0" containerName="authorino" containerID="cri-o://fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53" gracePeriod=30 Jul 12 08:17:06.566473 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.566447 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" event={"ID":"88cf8b34-f4bf-422d-bce6-1defb9bb2170","Type":"ContainerStarted","Data":"f79cd2d75a302c8c33449d4ef23d168ce7e5b47d3114b80acd52d0ee19f03443"} Jul 12 08:17:06.568802 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.568722 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" event={"ID":"0e2a6615-dab6-4bea-afcf-9ba07efd894d","Type":"ContainerStarted","Data":"8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008"} Jul 12 08:17:06.568802 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.568778 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" event={"ID":"0e2a6615-dab6-4bea-afcf-9ba07efd894d","Type":"ContainerStarted","Data":"3d26754c6761b7104c8837a9c93fbd771f3d55899ce1221bc779c9195a96b8a8"} Jul 12 08:17:06.569042 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.568865 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" podUID="0e2a6615-dab6-4bea-afcf-9ba07efd894d" containerName="authorino" containerID="cri-o://8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008" gracePeriod=30 Jul 12 08:17:06.579640 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.579599 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" podStartSLOduration=1.272666212 podStartE2EDuration="1.579585612s" totalImagesPullingTime="306.9194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:17:06.578506953 +0000 UTC m=+611.023992085" watchObservedRunningTime="2026-07-12 08:17:06.579585612 +0000 UTC m=+611.025070743" Jul 12 08:17:06.592557 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.592518 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" podStartSLOduration=1.240093168 podStartE2EDuration="1.592506095s" totalImagesPullingTime="352.412927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:17:06.591362508 +0000 UTC m=+611.036847639" watchObservedRunningTime="2026-07-12 08:17:06.592506095 +0000 UTC m=+611.037991234" Jul 12 08:17:06.843342 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.843316 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:06.846116 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.846098 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:06.897500 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.897479 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-484ts\" (UniqueName: \"kubernetes.io/projected/0e2a6615-dab6-4bea-afcf-9ba07efd894d-kube-api-access-484ts\") pod \"0e2a6615-dab6-4bea-afcf-9ba07efd894d\" (UID: \"0e2a6615-dab6-4bea-afcf-9ba07efd894d\") " Jul 12 08:17:06.897571 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.897511 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt9xw\" (UniqueName: \"kubernetes.io/projected/42b58e27-70f4-4d1d-8b54-2c5db60270b0-kube-api-access-dt9xw\") pod \"42b58e27-70f4-4d1d-8b54-2c5db60270b0\" (UID: \"42b58e27-70f4-4d1d-8b54-2c5db60270b0\") " Jul 12 08:17:06.899481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.899454 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e2a6615-dab6-4bea-afcf-9ba07efd894d-kube-api-access-484ts" (OuterVolumeSpecName: "kube-api-access-484ts") pod "0e2a6615-dab6-4bea-afcf-9ba07efd894d" (UID: "0e2a6615-dab6-4bea-afcf-9ba07efd894d"). InnerVolumeSpecName "kube-api-access-484ts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:17:06.899481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.899462 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b58e27-70f4-4d1d-8b54-2c5db60270b0-kube-api-access-dt9xw" (OuterVolumeSpecName: "kube-api-access-dt9xw") pod "42b58e27-70f4-4d1d-8b54-2c5db60270b0" (UID: "42b58e27-70f4-4d1d-8b54-2c5db60270b0"). InnerVolumeSpecName "kube-api-access-dt9xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:17:06.998146 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.998101 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-484ts\" (UniqueName: \"kubernetes.io/projected/0e2a6615-dab6-4bea-afcf-9ba07efd894d-kube-api-access-484ts\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:17:06.998146 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:06.998144 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt9xw\" (UniqueName: \"kubernetes.io/projected/42b58e27-70f4-4d1d-8b54-2c5db60270b0-kube-api-access-dt9xw\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:17:07.574875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.574837 2558 generic.go:356] "Generic (PLEG): container finished" podID="0e2a6615-dab6-4bea-afcf-9ba07efd894d" containerID="8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008" exitCode=0 Jul 12 08:17:07.575350 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.574886 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" Jul 12 08:17:07.575350 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.574931 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" event={"ID":"0e2a6615-dab6-4bea-afcf-9ba07efd894d","Type":"ContainerDied","Data":"8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008"} Jul 12 08:17:07.575350 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.574983 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57bf79ccd6-49pcv" event={"ID":"0e2a6615-dab6-4bea-afcf-9ba07efd894d","Type":"ContainerDied","Data":"3d26754c6761b7104c8837a9c93fbd771f3d55899ce1221bc779c9195a96b8a8"} Jul 12 08:17:07.575350 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.575007 2558 scope.go:117] "RemoveContainer" containerID="8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008" Jul 12 08:17:07.576354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.576328 2558 generic.go:356] "Generic (PLEG): container finished" podID="42b58e27-70f4-4d1d-8b54-2c5db60270b0" containerID="fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53" exitCode=0 Jul 12 08:17:07.576469 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.576360 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" Jul 12 08:17:07.576469 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.576411 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" event={"ID":"42b58e27-70f4-4d1d-8b54-2c5db60270b0","Type":"ContainerDied","Data":"fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53"} Jul 12 08:17:07.576469 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.576457 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vsmr7" event={"ID":"42b58e27-70f4-4d1d-8b54-2c5db60270b0","Type":"ContainerDied","Data":"91c7698175a71f70824fba32b118782a5cf3a0388d6f58f341927429b35bb75f"} Jul 12 08:17:07.578035 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.578011 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" event={"ID":"88cf8b34-f4bf-422d-bce6-1defb9bb2170","Type":"ContainerStarted","Data":"d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197"} Jul 12 08:17:07.584884 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.584831 2558 scope.go:117] "RemoveContainer" containerID="8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008" Jul 12 08:17:07.588881 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:17:07.588092 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008\": container with ID starting with 8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008 not found: ID does not exist" containerID="8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008" Jul 12 08:17:07.588881 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.588138 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008"} err="failed to get container status \"8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008\": rpc error: code = NotFound desc = could not find container \"8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008\": container with ID starting with 8cd74a3d472276c0ace1266c1aeb6e9b275fc2da645aa615d3ad058a7e72d008 not found: ID does not exist" Jul 12 08:17:07.588881 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.588163 2558 scope.go:117] "RemoveContainer" containerID="fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53" Jul 12 08:17:07.608771 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.607735 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" podStartSLOduration=2.176126805 podStartE2EDuration="2.60770747s" totalImagesPullingTime="431.580665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:17:07.607586132 +0000 UTC m=+612.053071263" watchObservedRunningTime="2026-07-12 08:17:07.60770747 +0000 UTC m=+612.053192602" Jul 12 08:17:07.622116 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.622068 2558 scope.go:117] "RemoveContainer" containerID="fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53" Jul 12 08:17:07.622568 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:17:07.622531 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53\": container with ID starting with fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53 not found: ID does not exist" containerID="fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53" Jul 12 08:17:07.622686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.622574 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53"} err="failed to get container status \"fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53\": rpc error: code = NotFound desc = could not find container \"fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53\": container with ID starting with fd93e5d8219530c6cf6db6ceef7f4e456c70862623b38729402422d33c557d53 not found: ID does not exist" Jul 12 08:17:07.628718 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.628652 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57bf79ccd6-49pcv"] Jul 12 08:17:07.630838 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.630813 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57bf79ccd6-49pcv"] Jul 12 08:17:07.640773 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.640723 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vsmr7"] Jul 12 08:17:07.642473 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:07.642448 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vsmr7"] Jul 12 08:17:08.387348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:08.387310 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e2a6615-dab6-4bea-afcf-9ba07efd894d" path="/var/lib/kubelet/pods/0e2a6615-dab6-4bea-afcf-9ba07efd894d/volumes" Jul 12 08:17:08.387630 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:08.387617 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b58e27-70f4-4d1d-8b54-2c5db60270b0" path="/var/lib/kubelet/pods/42b58e27-70f4-4d1d-8b54-2c5db60270b0/volumes" Jul 12 08:17:11.940934 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:11.940898 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:12.040381 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:12.040350 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:12.091507 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:12.091476 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:12.406938 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:12.406906 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:15.517051 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:15.517020 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:15.630368 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:15.630342 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:33.793793 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:33.793696 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:47.619538 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:47.619508 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:47.948607 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:47.948518 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:48.361963 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:48.361928 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:48.648241 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:48.648167 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:48.833234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:48.833205 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:48.930635 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:48.930553 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:56.777482 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:56.777447 2558 generic.go:356] "Generic (PLEG): container finished" podID="decfcf79-5029-46b2-8a3b-9d70fbfc5ee3" containerID="ff4872fa6199e95388b209a89cffe012859cc53a6cb3b2ad377de1af87aa68fb" exitCode=0 Jul 12 08:17:56.777939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:56.777521 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" event={"ID":"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3","Type":"ContainerDied","Data":"ff4872fa6199e95388b209a89cffe012859cc53a6cb3b2ad377de1af87aa68fb"} Jul 12 08:17:56.777939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:56.777926 2558 scope.go:117] "RemoveContainer" containerID="ff4872fa6199e95388b209a89cffe012859cc53a6cb3b2ad377de1af87aa68fb" Jul 12 08:17:56.778847 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:56.778819 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:17:56.778941 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:56.778886 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:17:56.778941 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:56.778918 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:17:57.783107 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:57.783073 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" event={"ID":"decfcf79-5029-46b2-8a3b-9d70fbfc5ee3","Type":"ContainerStarted","Data":"3ac6272f4e4ee65a9170947301638eaaadc9fbde6e27b490cdb7afc1ba2c4d2e"} Jul 12 08:17:58.284734 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:58.284689 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:17:58.285733 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:58.285712 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 12 08:17:58.285817 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:58.285792 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" podUID="decfcf79-5029-46b2-8a3b-9d70fbfc5ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 12 08:17:58.719359 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:58.719324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:58.815005 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:58.814973 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:17:59.284383 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:59.284349 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 12 08:17:59.284548 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:17:59.284415 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" podUID="decfcf79-5029-46b2-8a3b-9d70fbfc5ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 12 08:18:00.284248 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:00.284206 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 12 08:18:00.284614 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:00.284280 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" podUID="decfcf79-5029-46b2-8a3b-9d70fbfc5ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 12 08:18:00.363210 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:00.363173 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:18:01.284896 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:01.284854 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 12 08:18:01.285372 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:01.284951 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" podUID="decfcf79-5029-46b2-8a3b-9d70fbfc5ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 12 08:18:02.288621 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:02.288588 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:18:02.289928 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:02.289908 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz94c" Jul 12 08:18:16.309621 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:16.309586 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:25.534530 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:25.534492 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:25.608457 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:25.608429 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:26.025344 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:26.025309 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:29.029780 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:29.029722 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:34.079208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079170 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92"] Jul 12 08:18:34.079655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079500 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b58e27-70f4-4d1d-8b54-2c5db60270b0" containerName="authorino" Jul 12 08:18:34.079655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079511 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b58e27-70f4-4d1d-8b54-2c5db60270b0" containerName="authorino" Jul 12 08:18:34.079655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079520 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2a6615-dab6-4bea-afcf-9ba07efd894d" containerName="authorino" Jul 12 08:18:34.079655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079526 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2a6615-dab6-4bea-afcf-9ba07efd894d" containerName="authorino" Jul 12 08:18:34.079655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079590 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b58e27-70f4-4d1d-8b54-2c5db60270b0" containerName="authorino" Jul 12 08:18:34.079655 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.079598 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e2a6615-dab6-4bea-afcf-9ba07efd894d" containerName="authorino" Jul 12 08:18:34.082441 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.082422 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.085853 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.085824 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vxrzn\"" Jul 12 08:18:34.085979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.085826 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 12 08:18:34.085979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.085885 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 08:18:34.085979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.085837 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 08:18:34.092807 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.092786 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92"] Jul 12 08:18:34.111481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.111455 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:34.230082 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.230041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrj8t\" (UniqueName: \"kubernetes.io/projected/9c16538c-64fe-4675-a8a6-1b6352ca5323-kube-api-access-hrj8t\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.230082 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.230086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.230279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.230108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.230279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.230186 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.230279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.230214 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.230279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.230241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c16538c-64fe-4675-a8a6-1b6352ca5323-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331310 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c16538c-64fe-4675-a8a6-1b6352ca5323-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331372 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrj8t\" (UniqueName: \"kubernetes.io/projected/9c16538c-64fe-4675-a8a6-1b6352ca5323-kube-api-access-hrj8t\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331396 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331418 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331723 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331912 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331804 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.331912 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.331857 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.333735 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.333710 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c16538c-64fe-4675-a8a6-1b6352ca5323-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.334023 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.334004 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c16538c-64fe-4675-a8a6-1b6352ca5323-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.340372 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.340341 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrj8t\" (UniqueName: \"kubernetes.io/projected/9c16538c-64fe-4675-a8a6-1b6352ca5323-kube-api-access-hrj8t\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92\" (UID: \"9c16538c-64fe-4675-a8a6-1b6352ca5323\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.392912 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.392875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:34.523128 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.523102 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92"] Jul 12 08:18:34.525548 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:18:34.525523 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c16538c_64fe_4675_a8a6_1b6352ca5323.slice/crio-cab5198d19adfa424c06efe59d196f36f76c0973baa2d4ca452918bcf4de027c WatchSource:0}: Error finding container cab5198d19adfa424c06efe59d196f36f76c0973baa2d4ca452918bcf4de027c: Status 404 returned error can't find the container with id cab5198d19adfa424c06efe59d196f36f76c0973baa2d4ca452918bcf4de027c Jul 12 08:18:34.526733 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.526717 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:18:34.936011 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:34.935970 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" event={"ID":"9c16538c-64fe-4675-a8a6-1b6352ca5323","Type":"ContainerStarted","Data":"cab5198d19adfa424c06efe59d196f36f76c0973baa2d4ca452918bcf4de027c"} Jul 12 08:18:35.942033 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:35.941996 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" event={"ID":"9c16538c-64fe-4675-a8a6-1b6352ca5323","Type":"ContainerStarted","Data":"b463032d7b7c3016e22631577ce1991eeca7986fc7644b3608dcb57cfd8ad508"} Jul 12 08:18:44.393033 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:44.392999 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:44.393397 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:44.393056 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:44.405400 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:44.405368 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:44.426876 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:44.426829 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" podStartSLOduration=9.453794176 podStartE2EDuration="10.426815797s" totalImagesPullingTime="973.021621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:18:35.980518718 +0000 UTC m=+700.426003847" watchObservedRunningTime="2026-07-12 08:18:44.426815797 +0000 UTC m=+708.872300953" Jul 12 08:18:44.990035 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:44.990002 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92" Jul 12 08:18:45.261967 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:45.261884 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:45.327292 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:45.327249 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:45.854080 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:45.854048 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:55.617624 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.617548 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj"] Jul 12 08:18:55.620660 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.620641 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.623327 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.623310 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 12 08:18:55.641653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.641632 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj"] Jul 12 08:18:55.711191 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.711157 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.711348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.711195 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.711348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.711224 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc326c5-7428-4f51-af73-2a6058c07bb9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.711348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.711260 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.711348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.711281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.711348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.711301 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gjcm\" (UniqueName: \"kubernetes.io/projected/2bc326c5-7428-4f51-af73-2a6058c07bb9-kube-api-access-4gjcm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.728900 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.728873 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:18:55.812249 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812249 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812254 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812499 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812273 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc326c5-7428-4f51-af73-2a6058c07bb9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812499 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812383 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812499 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812418 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812659 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812569 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gjcm\" (UniqueName: \"kubernetes.io/projected/2bc326c5-7428-4f51-af73-2a6058c07bb9-kube-api-access-4gjcm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812711 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812677 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812801 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812716 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.812974 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.812956 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.814755 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.814714 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc326c5-7428-4f51-af73-2a6058c07bb9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.815187 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.815170 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc326c5-7428-4f51-af73-2a6058c07bb9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.823273 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.823250 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gjcm\" (UniqueName: \"kubernetes.io/projected/2bc326c5-7428-4f51-af73-2a6058c07bb9-kube-api-access-4gjcm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj\" (UID: \"2bc326c5-7428-4f51-af73-2a6058c07bb9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:55.929918 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:55.929837 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:18:56.069963 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:56.069939 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj"] Jul 12 08:18:56.071714 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:18:56.071687 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bc326c5_7428_4f51_af73_2a6058c07bb9.slice/crio-9fb30a8cb4594e54701d0b9b5f2a56ba046043fb21fde204f57688753afdb515 WatchSource:0}: Error finding container 9fb30a8cb4594e54701d0b9b5f2a56ba046043fb21fde204f57688753afdb515: Status 404 returned error can't find the container with id 9fb30a8cb4594e54701d0b9b5f2a56ba046043fb21fde204f57688753afdb515 Jul 12 08:18:57.026858 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:57.026817 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" event={"ID":"2bc326c5-7428-4f51-af73-2a6058c07bb9","Type":"ContainerStarted","Data":"14069ad239decb9f9d5ebd259aafa01ce05bf04f1abc69a8ab4f89f01fda1f4b"} Jul 12 08:18:57.027221 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:57.026868 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" event={"ID":"2bc326c5-7428-4f51-af73-2a6058c07bb9","Type":"ContainerStarted","Data":"9fb30a8cb4594e54701d0b9b5f2a56ba046043fb21fde204f57688753afdb515"} Jul 12 08:18:57.055004 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:18:57.054955 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" podStartSLOduration=1.886129816 podStartE2EDuration="2.054938443s" totalImagesPullingTime="168.808627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:18:57.053438053 +0000 UTC m=+721.498923184" watchObservedRunningTime="2026-07-12 08:18:57.054938443 +0000 UTC m=+721.500423630" Jul 12 08:19:04.006564 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:04.006527 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:19:05.930502 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:05.930468 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:19:05.930979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:05.930520 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:19:05.943199 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:05.943172 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:19:06.073075 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:06.073044 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj" Jul 12 08:19:50.769582 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.769540 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b4867c6ff-hzlwv"] Jul 12 08:19:50.773082 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.773059 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:50.782124 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.782097 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b4867c6ff-hzlwv"] Jul 12 08:19:50.838590 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.838558 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jmzg\" (UniqueName: \"kubernetes.io/projected/c9093c26-8655-475a-8fba-01d6b174a34a-kube-api-access-7jmzg\") pod \"authorino-b4867c6ff-hzlwv\" (UID: \"c9093c26-8655-475a-8fba-01d6b174a34a\") " pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:50.838590 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.838589 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9093c26-8655-475a-8fba-01d6b174a34a-tls-cert\") pod \"authorino-b4867c6ff-hzlwv\" (UID: \"c9093c26-8655-475a-8fba-01d6b174a34a\") " pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:50.939812 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.939784 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jmzg\" (UniqueName: \"kubernetes.io/projected/c9093c26-8655-475a-8fba-01d6b174a34a-kube-api-access-7jmzg\") pod \"authorino-b4867c6ff-hzlwv\" (UID: \"c9093c26-8655-475a-8fba-01d6b174a34a\") " pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:50.939970 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.939820 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9093c26-8655-475a-8fba-01d6b174a34a-tls-cert\") pod \"authorino-b4867c6ff-hzlwv\" (UID: \"c9093c26-8655-475a-8fba-01d6b174a34a\") " pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:50.942217 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.942194 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9093c26-8655-475a-8fba-01d6b174a34a-tls-cert\") pod \"authorino-b4867c6ff-hzlwv\" (UID: \"c9093c26-8655-475a-8fba-01d6b174a34a\") " pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:50.947510 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:50.947480 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jmzg\" (UniqueName: \"kubernetes.io/projected/c9093c26-8655-475a-8fba-01d6b174a34a-kube-api-access-7jmzg\") pod \"authorino-b4867c6ff-hzlwv\" (UID: \"c9093c26-8655-475a-8fba-01d6b174a34a\") " pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:51.083032 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:51.083003 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b4867c6ff-hzlwv" Jul 12 08:19:51.207037 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:51.207015 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b4867c6ff-hzlwv"] Jul 12 08:19:51.209198 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:19:51.209175 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9093c26_8655_475a_8fba_01d6b174a34a.slice/crio-705dc7738888fd44f6a8619d184b606087d0cfd0889838f928a92532b249afcd WatchSource:0}: Error finding container 705dc7738888fd44f6a8619d184b606087d0cfd0889838f928a92532b249afcd: Status 404 returned error can't find the container with id 705dc7738888fd44f6a8619d184b606087d0cfd0889838f928a92532b249afcd Jul 12 08:19:51.249132 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:51.249099 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b4867c6ff-hzlwv" event={"ID":"c9093c26-8655-475a-8fba-01d6b174a34a","Type":"ContainerStarted","Data":"705dc7738888fd44f6a8619d184b606087d0cfd0889838f928a92532b249afcd"} Jul 12 08:19:52.256496 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.256454 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b4867c6ff-hzlwv" event={"ID":"c9093c26-8655-475a-8fba-01d6b174a34a","Type":"ContainerStarted","Data":"f13f724cc405521e2c2f8eb06dd3b0923fa77c16fa64091441f457dd9b3fc5f8"} Jul 12 08:19:52.276714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.276662 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b4867c6ff-hzlwv" podStartSLOduration=1.7265375870000002 podStartE2EDuration="2.276644803s" totalImagesPullingTime="550.107216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:19:52.27553925 +0000 UTC m=+776.721024381" watchObservedRunningTime="2026-07-12 08:19:52.276644803 +0000 UTC m=+776.722129933" Jul 12 08:19:52.306173 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.306131 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dbd48c4c8-jvqw2"] Jul 12 08:19:52.306405 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.306365 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" podUID="88cf8b34-f4bf-422d-bce6-1defb9bb2170" containerName="authorino" containerID="cri-o://d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197" gracePeriod=30 Jul 12 08:19:52.558100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.558072 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:19:52.655826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.655791 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88cf8b34-f4bf-422d-bce6-1defb9bb2170-tls-cert\") pod \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " Jul 12 08:19:52.656003 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.655899 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr66h\" (UniqueName: \"kubernetes.io/projected/88cf8b34-f4bf-422d-bce6-1defb9bb2170-kube-api-access-vr66h\") pod \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\" (UID: \"88cf8b34-f4bf-422d-bce6-1defb9bb2170\") " Jul 12 08:19:52.658272 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.658234 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88cf8b34-f4bf-422d-bce6-1defb9bb2170-kube-api-access-vr66h" (OuterVolumeSpecName: "kube-api-access-vr66h") pod "88cf8b34-f4bf-422d-bce6-1defb9bb2170" (UID: "88cf8b34-f4bf-422d-bce6-1defb9bb2170"). InnerVolumeSpecName "kube-api-access-vr66h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:19:52.665620 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.665591 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88cf8b34-f4bf-422d-bce6-1defb9bb2170-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "88cf8b34-f4bf-422d-bce6-1defb9bb2170" (UID: "88cf8b34-f4bf-422d-bce6-1defb9bb2170"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:19:52.757096 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.757063 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vr66h\" (UniqueName: \"kubernetes.io/projected/88cf8b34-f4bf-422d-bce6-1defb9bb2170-kube-api-access-vr66h\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:19:52.757096 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.757090 2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88cf8b34-f4bf-422d-bce6-1defb9bb2170-tls-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:19:52.791505 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.791423 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:19:52.893259 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:52.893229 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:19:53.262090 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.262054 2558 generic.go:356] "Generic (PLEG): container finished" podID="88cf8b34-f4bf-422d-bce6-1defb9bb2170" containerID="d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197" exitCode=0 Jul 12 08:19:53.262533 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.262110 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" Jul 12 08:19:53.262533 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.262134 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" event={"ID":"88cf8b34-f4bf-422d-bce6-1defb9bb2170","Type":"ContainerDied","Data":"d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197"} Jul 12 08:19:53.262533 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.262190 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dbd48c4c8-jvqw2" event={"ID":"88cf8b34-f4bf-422d-bce6-1defb9bb2170","Type":"ContainerDied","Data":"f79cd2d75a302c8c33449d4ef23d168ce7e5b47d3114b80acd52d0ee19f03443"} Jul 12 08:19:53.262533 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.262208 2558 scope.go:117] "RemoveContainer" containerID="d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197" Jul 12 08:19:53.270447 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.270430 2558 scope.go:117] "RemoveContainer" containerID="d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197" Jul 12 08:19:53.270718 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:19:53.270698 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197\": container with ID starting with d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197 not found: ID does not exist" containerID="d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197" Jul 12 08:19:53.270786 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.270727 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197"} err="failed to get container status \"d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197\": rpc error: code = NotFound desc = could not find container \"d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197\": container with ID starting with d6d9aa9d99e6ed865b80bb1ddfaa3d5d265e8cbe95821230f6b24d8ef812d197 not found: ID does not exist" Jul 12 08:19:53.286877 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.286848 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dbd48c4c8-jvqw2"] Jul 12 08:19:53.291435 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.291406 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dbd48c4c8-jvqw2"] Jul 12 08:19:53.917380 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:53.917347 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:19:54.014884 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:54.014851 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:19:54.387417 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:19:54.387383 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88cf8b34-f4bf-422d-bce6-1defb9bb2170" path="/var/lib/kubelet/pods/88cf8b34-f4bf-422d-bce6-1defb9bb2170/volumes" Jul 12 08:21:04.274914 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:04.274861 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:05.809097 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:05.809060 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:44.680999 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:44.680953 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:44.780054 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:44.780022 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:44.892849 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:44.892813 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:50.571973 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:50.571892 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:50.630223 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:50.630156 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:50.726343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:50.726307 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:50.826609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:50.826529 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:50.926608 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:50.926577 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:56.135959 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:56.135930 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:21:56.137175 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:56.137157 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:21:56.599313 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:56.599281 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:56.660539 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:21:56.660507 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:49.653870 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:23:49.653776 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:21.700529 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:26:21.700454 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:21.787762 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:26:21.787718 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:56.158933 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:26:56.158901 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:26:56.160789 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:26:56.160768 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:27:06.117155 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:27:06.117097 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:04.021944 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:28:04.021901 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:29.743433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:29:29.743344 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:30.965883 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:29:30.965849 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:56.181011 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:31:56.180976 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:31:56.183722 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:31:56.183703 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:32:45.578689 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:45.578617 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:45.668708 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:45.668677 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:50.905202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:50.905167 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:51.004722 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.004690 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:51.099003 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.098734 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw"] Jul 12 08:32:51.099233 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.099212 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88cf8b34-f4bf-422d-bce6-1defb9bb2170" containerName="authorino" Jul 12 08:32:51.099295 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.099238 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="88cf8b34-f4bf-422d-bce6-1defb9bb2170" containerName="authorino" Jul 12 08:32:51.099352 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.099340 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="88cf8b34-f4bf-422d-bce6-1defb9bb2170" containerName="authorino" Jul 12 08:32:51.102586 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.102567 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103289 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-data\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103434 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103324 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-podinfo\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103434 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103346 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-certs\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103525 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103459 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkkbb\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-kube-api-access-bkkbb\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103525 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103504 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e9fc00-7679-460c-9cff-ddf64fe62516-istiod-ca-cert\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103543 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-envoy\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-token\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103840 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-credential-socket\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.103840 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.103736 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-socket\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.104910 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.104886 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-836b156d-openshift-default-dockercfg-2pn86\"" Jul 12 08:32:51.116312 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.116291 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw"] Jul 12 08:32:51.127882 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.127853 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:51.204584 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204506 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-credential-socket\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.204584 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204541 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-socket\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.204584 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204563 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-data\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.204869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204591 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-podinfo\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.204869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204612 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-certs\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.204869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204754 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkbb\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-kube-api-access-bkkbb\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.204869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204817 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e9fc00-7679-460c-9cff-ddf64fe62516-istiod-ca-cert\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204892 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-envoy\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204950 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-credential-socket\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-token\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.204993 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-socket\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.205041 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-data\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205078 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.205076 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-certs\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.205582 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.205564 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e9fc00-7679-460c-9cff-ddf64fe62516-istiod-ca-cert\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.206974 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.206951 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-envoy\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.207197 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.207179 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-podinfo\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.214432 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.214409 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkkbb\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-kube-api-access-bkkbb\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.214624 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.214604 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-token\") pod \"e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.228318 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.228296 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:51.412955 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.412927 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:51.541176 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.541074 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw"] Jul 12 08:32:51.543856 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:32:51.543831 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e9fc00_7679_460c_9cff_ddf64fe62516.slice/crio-e2905bb81966912e1ac4b0ded1879d766887b52192a249bbb6f9675c8ec8416f WatchSource:0}: Error finding container e2905bb81966912e1ac4b0ded1879d766887b52192a249bbb6f9675c8ec8416f: Status 404 returned error can't find the container with id e2905bb81966912e1ac4b0ded1879d766887b52192a249bbb6f9675c8ec8416f Jul 12 08:32:51.545931 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.545902 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:32:51.545987 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.545958 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:32:51.546022 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:51.545986 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:32:52.287350 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.287288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:52.408601 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.408572 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:52.478251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.478215 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" event={"ID":"67e9fc00-7679-460c-9cff-ddf64fe62516","Type":"ContainerStarted","Data":"26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f"} Jul 12 08:32:52.478420 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.478281 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" event={"ID":"67e9fc00-7679-460c-9cff-ddf64fe62516","Type":"ContainerStarted","Data":"e2905bb81966912e1ac4b0ded1879d766887b52192a249bbb6f9675c8ec8416f"} Jul 12 08:32:52.501503 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.501457 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" podStartSLOduration=1.5014403509999998 podStartE2EDuration="1.501440351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:32:52.500184702 +0000 UTC m=+1556.945669858" watchObservedRunningTime="2026-07-12 08:32:52.501440351 +0000 UTC m=+1556.946925481" Jul 12 08:32:52.525502 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.525474 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:52.661185 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.661153 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:52.991727 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:52.991613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:53.095850 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:53.095811 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:53.413276 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:53.413244 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:53.418088 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:53.418063 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:53.482715 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:53.482681 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:53.483714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:53.483688 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:32:54.603386 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:54.603359 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:57.279402 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:57.279369 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:57.445916 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:57.445880 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw"] Jul 12 08:32:57.446164 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:57.446118 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" podUID="67e9fc00-7679-460c-9cff-ddf64fe62516" containerName="istio-proxy" containerID="cri-o://26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f" gracePeriod=30 Jul 12 08:32:57.471167 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:57.471134 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:57.570925 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:57.570869 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:57.631468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:57.631356 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:59.278296 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:32:59.278270 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:03.088283 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.088258 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:33:03.210788 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210686 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e9fc00-7679-460c-9cff-ddf64fe62516-istiod-ca-cert\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.210788 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210727 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkkbb\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-kube-api-access-bkkbb\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.210788 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210765 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-envoy\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210881 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-data\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210916 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-token\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210951 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-credential-socket\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.210984 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-podinfo\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211016 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-certs\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211249 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211196 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e9fc00-7679-460c-9cff-ddf64fe62516-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:33:03.211306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211292 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:03.211380 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211349 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:03.211453 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211386 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-socket\") pod \"67e9fc00-7679-460c-9cff-ddf64fe62516\" (UID: \"67e9fc00-7679-460c-9cff-ddf64fe62516\") " Jul 12 08:33:03.211657 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211620 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:03.211796 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211670 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.211796 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211689 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e9fc00-7679-460c-9cff-ddf64fe62516-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.211796 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211712 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.211975 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.211809 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-data" (OuterVolumeSpecName: "istio-data") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:03.213259 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.213224 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-token" (OuterVolumeSpecName: "istio-token") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:03.213368 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.213265 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:03.213368 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.213263 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-kube-api-access-bkkbb" (OuterVolumeSpecName: "kube-api-access-bkkbb") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "kube-api-access-bkkbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:03.213368 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.213335 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67e9fc00-7679-460c-9cff-ddf64fe62516" (UID: "67e9fc00-7679-460c-9cff-ddf64fe62516"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:33:03.312381 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.312345 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkkbb\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-kube-api-access-bkkbb\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.312381 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.312373 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.312381 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.312385 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.312595 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.312396 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.312595 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.312405 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e9fc00-7679-460c-9cff-ddf64fe62516-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.312595 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.312415 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e9fc00-7679-460c-9cff-ddf64fe62516-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:03.498085 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.498012 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:03.528616 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.528585 2558 generic.go:356] "Generic (PLEG): container finished" podID="67e9fc00-7679-460c-9cff-ddf64fe62516" containerID="26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f" exitCode=0 Jul 12 08:33:03.528760 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.528627 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" Jul 12 08:33:03.528760 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.528642 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" event={"ID":"67e9fc00-7679-460c-9cff-ddf64fe62516","Type":"ContainerDied","Data":"26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f"} Jul 12 08:33:03.528760 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.528682 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw" event={"ID":"67e9fc00-7679-460c-9cff-ddf64fe62516","Type":"ContainerDied","Data":"e2905bb81966912e1ac4b0ded1879d766887b52192a249bbb6f9675c8ec8416f"} Jul 12 08:33:03.528760 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.528703 2558 scope.go:117] "RemoveContainer" containerID="26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f" Jul 12 08:33:03.536801 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.536783 2558 scope.go:117] "RemoveContainer" containerID="26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f" Jul 12 08:33:03.537051 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:33:03.537033 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f\": container with ID starting with 26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f not found: ID does not exist" containerID="26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f" Jul 12 08:33:03.537110 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.537063 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f"} err="failed to get container status \"26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f\": rpc error: code = NotFound desc = could not find container \"26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f\": container with ID starting with 26b2023e38030cb384c60f9ca6d446be3869c6a8766b74468edbec9d3c51c70f not found: ID does not exist" Jul 12 08:33:03.549775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.549736 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw"] Jul 12 08:33:03.556426 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.556404 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-836b156d-openshift-default-79bfbc65b5-ch4mw"] Jul 12 08:33:03.568001 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.567958 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:03.657098 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.657066 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4"] Jul 12 08:33:03.657503 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.657485 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e9fc00-7679-460c-9cff-ddf64fe62516" containerName="istio-proxy" Jul 12 08:33:03.657579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.657507 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e9fc00-7679-460c-9cff-ddf64fe62516" containerName="istio-proxy" Jul 12 08:33:03.657579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.657562 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e9fc00-7679-460c-9cff-ddf64fe62516" containerName="istio-proxy" Jul 12 08:33:03.661604 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.661581 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.664083 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.664058 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-65d96a1b-openshift-default-dockercfg-tt9cd\"" Jul 12 08:33:03.673382 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.673332 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4"] Jul 12 08:33:03.707072 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.707047 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:03.716239 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716212 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-podinfo\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istiod-ca-cert\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716271 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-credential-socket\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-token\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716357 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-socket\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716378 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-envoy\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqsg\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-kube-api-access-bgqsg\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716568 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716469 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-data\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.716568 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.716514 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-certs\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.768029 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.767946 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:03.817265 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-podinfo\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817265 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817266 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istiod-ca-cert\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817288 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-credential-socket\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817321 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-token\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817346 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-socket\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817368 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-envoy\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqsg\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-kube-api-access-bgqsg\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-data\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817731 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817474 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-certs\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817879 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-certs\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.817964 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817940 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-socket\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.818017 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.817972 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-credential-socket\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.818068 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.818042 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-data\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.818324 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.818304 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istiod-ca-cert\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.819664 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.819641 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-envoy\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.819915 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.819897 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-podinfo\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.825249 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.825230 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqsg\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-kube-api-access-bgqsg\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.825658 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.825642 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-token\") pod \"e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:03.974041 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:03.974004 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:04.104403 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.104253 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4"] Jul 12 08:33:04.107279 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:33:04.107243 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f50389_4ff6_4f97_840a_a55705cb8b0b.slice/crio-473bd22bee0169087332fc7ba0c979e72cd00b532d92222fc0427967eaef58ef WatchSource:0}: Error finding container 473bd22bee0169087332fc7ba0c979e72cd00b532d92222fc0427967eaef58ef: Status 404 returned error can't find the container with id 473bd22bee0169087332fc7ba0c979e72cd00b532d92222fc0427967eaef58ef Jul 12 08:33:04.109919 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.109884 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:33:04.110027 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.109967 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:33:04.110027 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.110011 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:33:04.388186 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.388145 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e9fc00-7679-460c-9cff-ddf64fe62516" path="/var/lib/kubelet/pods/67e9fc00-7679-460c-9cff-ddf64fe62516/volumes" Jul 12 08:33:04.534956 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.534923 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" event={"ID":"f2f50389-4ff6-4f97-840a-a55705cb8b0b","Type":"ContainerStarted","Data":"7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301"} Jul 12 08:33:04.535102 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.534980 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" event={"ID":"f2f50389-4ff6-4f97-840a-a55705cb8b0b","Type":"ContainerStarted","Data":"473bd22bee0169087332fc7ba0c979e72cd00b532d92222fc0427967eaef58ef"} Jul 12 08:33:04.555810 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.555737 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" podStartSLOduration=1.555721717 podStartE2EDuration="1.555721717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:33:04.554863192 +0000 UTC m=+1569.000348322" watchObservedRunningTime="2026-07-12 08:33:04.555721717 +0000 UTC m=+1569.001206846" Jul 12 08:33:04.813621 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.813582 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:04.898827 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.898794 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:04.969267 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.969236 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:04.974202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:04.974168 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:05.070189 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:05.070084 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:05.456188 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:05.456087 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:05.539829 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:05.539778 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:05.979347 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:05.979315 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:06.543680 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:06.543653 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:06.544563 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:06.544543 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:06.973275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:06.973241 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:09.876620 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:09.876590 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:10.304243 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:10.304161 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:15.922153 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:15.922109 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4"] Jul 12 08:33:15.922671 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:15.922307 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" podUID="f2f50389-4ff6-4f97-840a-a55705cb8b0b" containerName="istio-proxy" containerID="cri-o://7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301" gracePeriod=30 Jul 12 08:33:15.944537 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:15.944493 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:16.041613 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:16.041572 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:17.166652 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.166623 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:17.239075 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239047 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-envoy\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239197 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239091 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-socket\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239197 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239140 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-data\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239197 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239160 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-token\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239197 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239179 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-podinfo\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239208 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgqsg\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-kube-api-access-bgqsg\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239264 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istiod-ca-cert\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239292 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-certs\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239329 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-credential-socket\") pod \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\" (UID: \"f2f50389-4ff6-4f97-840a-a55705cb8b0b\") " Jul 12 08:33:17.239588 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239465 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:17.239656 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239589 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.239707 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239691 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:17.239782 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239691 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:33:17.239844 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239802 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:17.240002 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.239982 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:17.241294 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.241263 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:17.241411 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.241366 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:17.241466 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.241452 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:33:17.241786 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.241768 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-kube-api-access-bgqsg" (OuterVolumeSpecName: "kube-api-access-bgqsg") pod "f2f50389-4ff6-4f97-840a-a55705cb8b0b" (UID: "f2f50389-4ff6-4f97-840a-a55705cb8b0b"). InnerVolumeSpecName "kube-api-access-bgqsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:17.340034 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340001 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340034 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340028 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340034 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340040 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340052 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340063 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340073 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340083 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2f50389-4ff6-4f97-840a-a55705cb8b0b-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.340262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.340094 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgqsg\" (UniqueName: \"kubernetes.io/projected/f2f50389-4ff6-4f97-840a-a55705cb8b0b-kube-api-access-bgqsg\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:17.543093 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.543062 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:17.589536 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.589503 2558 generic.go:356] "Generic (PLEG): container finished" podID="f2f50389-4ff6-4f97-840a-a55705cb8b0b" containerID="7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301" exitCode=0 Jul 12 08:33:17.589687 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.589571 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" Jul 12 08:33:17.589687 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.589592 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" event={"ID":"f2f50389-4ff6-4f97-840a-a55705cb8b0b","Type":"ContainerDied","Data":"7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301"} Jul 12 08:33:17.589687 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.589679 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4" event={"ID":"f2f50389-4ff6-4f97-840a-a55705cb8b0b","Type":"ContainerDied","Data":"473bd22bee0169087332fc7ba0c979e72cd00b532d92222fc0427967eaef58ef"} Jul 12 08:33:17.589823 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.589709 2558 scope.go:117] "RemoveContainer" containerID="7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301" Jul 12 08:33:17.598896 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.598885 2558 scope.go:117] "RemoveContainer" containerID="7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301" Jul 12 08:33:17.599168 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:33:17.599149 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301\": container with ID starting with 7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301 not found: ID does not exist" containerID="7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301" Jul 12 08:33:17.599231 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.599181 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301"} err="failed to get container status \"7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301\": rpc error: code = NotFound desc = could not find container \"7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301\": container with ID starting with 7a509c84a159350790d12b971f18f8a3241baff81bc5a5ff0546a8123d744301 not found: ID does not exist" Jul 12 08:33:17.609671 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.609648 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4"] Jul 12 08:33:17.615003 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:17.614968 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-65d96a1b-openshift-default-678d5477d7-ncgj4"] Jul 12 08:33:18.386714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:18.386673 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f50389-4ff6-4f97-840a-a55705cb8b0b" path="/var/lib/kubelet/pods/f2f50389-4ff6-4f97-840a-a55705cb8b0b/volumes" Jul 12 08:33:21.817677 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.817636 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:21.883621 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.883588 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:21.993121 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.993091 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq"] Jul 12 08:33:21.993418 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.993406 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f50389-4ff6-4f97-840a-a55705cb8b0b" containerName="istio-proxy" Jul 12 08:33:21.993465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.993420 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f50389-4ff6-4f97-840a-a55705cb8b0b" containerName="istio-proxy" Jul 12 08:33:21.993499 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.993480 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f50389-4ff6-4f97-840a-a55705cb8b0b" containerName="istio-proxy" Jul 12 08:33:21.997628 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:21.997605 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.000242 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.000223 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5249a3b1-openshift-default-dockercfg-rwdsf\"" Jul 12 08:33:22.010757 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.010721 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq"] Jul 12 08:33:22.037463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.037437 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:22.081380 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp9w9\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-kube-api-access-gp9w9\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081380 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081348 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-credential-socket\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081380 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081368 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17762cf-8189-4e73-a8c7-53508ce430df-istiod-ca-cert\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081555 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081422 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-socket\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081555 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-istio-token\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081555 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081510 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17762cf-8189-4e73-a8c7-53508ce430df-istio-podinfo\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-envoy\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081577 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-data\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.081666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.081602 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-certs\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.129602 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.129573 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:22.182518 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182489 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-istio-token\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182644 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182531 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17762cf-8189-4e73-a8c7-53508ce430df-istio-podinfo\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182764 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182720 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-envoy\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182813 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-data\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182861 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182817 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-certs\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182909 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182891 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp9w9\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-kube-api-access-gp9w9\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182945 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182916 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-credential-socket\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.182945 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182935 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17762cf-8189-4e73-a8c7-53508ce430df-istiod-ca-cert\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.183080 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.182954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-socket\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.183255 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.183226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-certs\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.183255 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.183242 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-data\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.183395 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.183291 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-socket\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.183666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.183639 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-credential-socket\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.183783 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.183680 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17762cf-8189-4e73-a8c7-53508ce430df-istiod-ca-cert\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.184940 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.184921 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-envoy\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.185138 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.185121 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17762cf-8189-4e73-a8c7-53508ce430df-istio-podinfo\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.194562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.194543 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp9w9\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-kube-api-access-gp9w9\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.194656 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.194635 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-istio-token\") pod \"e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.310938 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.310901 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:22.439253 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.439222 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq"] Jul 12 08:33:22.442522 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:33:22.442498 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17762cf_8189_4e73_a8c7_53508ce430df.slice/crio-3d89c64805b0b6041cb4ba5fabaf3394f281f940910c1a4ff6409693bd882bf8 WatchSource:0}: Error finding container 3d89c64805b0b6041cb4ba5fabaf3394f281f940910c1a4ff6409693bd882bf8: Status 404 returned error can't find the container with id 3d89c64805b0b6041cb4ba5fabaf3394f281f940910c1a4ff6409693bd882bf8 Jul 12 08:33:22.444600 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.444568 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:33:22.444693 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.444654 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:33:22.444772 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.444712 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:33:22.613477 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.613393 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" event={"ID":"f17762cf-8189-4e73-a8c7-53508ce430df","Type":"ContainerStarted","Data":"d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3"} Jul 12 08:33:22.613477 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.613458 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" event={"ID":"f17762cf-8189-4e73-a8c7-53508ce430df","Type":"ContainerStarted","Data":"3d89c64805b0b6041cb4ba5fabaf3394f281f940910c1a4ff6409693bd882bf8"} Jul 12 08:33:22.634895 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:22.634841 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" podStartSLOduration=1.634821073 podStartE2EDuration="1.634821073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:33:22.63251225 +0000 UTC m=+1587.077997384" watchObservedRunningTime="2026-07-12 08:33:22.634821073 +0000 UTC m=+1587.080306204" Jul 12 08:33:23.237382 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.237344 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:23.311854 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.311823 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:23.316432 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.316409 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:23.328100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.328069 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:23.408872 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.408819 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:23.619492 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.619465 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:23.620486 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.620469 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:23.713253 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.713220 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:23.812091 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.812061 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:23.914043 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:23.913939 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:25.108490 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:25.108453 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:27.565185 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:27.565152 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:27.745118 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:27.745084 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq"] Jul 12 08:33:27.745317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:27.745292 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" podUID="f17762cf-8189-4e73-a8c7-53508ce430df" containerName="istio-proxy" containerID="cri-o://d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3" gracePeriod=30 Jul 12 08:33:27.769844 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:27.769816 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:27.867826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:27.867708 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:27.932547 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:27.932494 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:29.561489 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:29.561454 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:33.835460 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.835358 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:33.891626 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.890487 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:33.920766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.920724 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:33.991653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991572 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-data\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991624 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp9w9\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-kube-api-access-gp9w9\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991685 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-credential-socket\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991706 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-envoy\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991730 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17762cf-8189-4e73-a8c7-53508ce430df-istiod-ca-cert\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991782 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-istio-token\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991808 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-certs\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991855 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-socket\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.991904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.991882 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17762cf-8189-4e73-a8c7-53508ce430df-istio-podinfo\") pod \"f17762cf-8189-4e73-a8c7-53508ce430df\" (UID: \"f17762cf-8189-4e73-a8c7-53508ce430df\") " Jul 12 08:33:33.992254 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.992060 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:33.992254 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.992155 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:33.992254 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.992152 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17762cf-8189-4e73-a8c7-53508ce430df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:33:33.992395 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.992353 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:33.992451 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.992395 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:33.992509 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.992438 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-data" (OuterVolumeSpecName: "istio-data") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:33.994033 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.994000 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-istio-token" (OuterVolumeSpecName: "istio-token") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:33.994143 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.994054 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-kube-api-access-gp9w9" (OuterVolumeSpecName: "kube-api-access-gp9w9") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "kube-api-access-gp9w9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:33.994295 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.994275 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:33:33.994333 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:33.994307 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f17762cf-8189-4e73-a8c7-53508ce430df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f17762cf-8189-4e73-a8c7-53508ce430df" (UID: "f17762cf-8189-4e73-a8c7-53508ce430df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:33:34.093574 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093538 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093574 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093569 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093574 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093579 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093589 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17762cf-8189-4e73-a8c7-53508ce430df-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093597 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093605 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp9w9\" (UniqueName: \"kubernetes.io/projected/f17762cf-8189-4e73-a8c7-53508ce430df-kube-api-access-gp9w9\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093615 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17762cf-8189-4e73-a8c7-53508ce430df-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.093831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.093624 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17762cf-8189-4e73-a8c7-53508ce430df-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:33:34.294784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.294680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:34.395407 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.395381 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:34.493058 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.493027 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:34.675612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.675582 2558 generic.go:356] "Generic (PLEG): container finished" podID="f17762cf-8189-4e73-a8c7-53508ce430df" containerID="d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3" exitCode=0 Jul 12 08:33:34.675784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.675645 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" Jul 12 08:33:34.675784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.675667 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" event={"ID":"f17762cf-8189-4e73-a8c7-53508ce430df","Type":"ContainerDied","Data":"d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3"} Jul 12 08:33:34.675784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.675728 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq" event={"ID":"f17762cf-8189-4e73-a8c7-53508ce430df","Type":"ContainerDied","Data":"3d89c64805b0b6041cb4ba5fabaf3394f281f940910c1a4ff6409693bd882bf8"} Jul 12 08:33:34.675784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.675766 2558 scope.go:117] "RemoveContainer" containerID="d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3" Jul 12 08:33:34.684083 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.684072 2558 scope.go:117] "RemoveContainer" containerID="d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3" Jul 12 08:33:34.684316 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:33:34.684300 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3\": container with ID starting with d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3 not found: ID does not exist" containerID="d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3" Jul 12 08:33:34.684365 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.684322 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3"} err="failed to get container status \"d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3\": rpc error: code = NotFound desc = could not find container \"d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3\": container with ID starting with d37aea35a695a4527230717a963a55776a60fb0844b34f868a705c2e6c86bda3 not found: ID does not exist" Jul 12 08:33:34.692579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.692554 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq"] Jul 12 08:33:34.699494 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:34.699472 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5249a3b1-openshift-default-64b8cb69f5-nc4hq"] Jul 12 08:33:35.794737 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:35.794700 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:36.386619 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:36.386585 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f17762cf-8189-4e73-a8c7-53508ce430df" path="/var/lib/kubelet/pods/f17762cf-8189-4e73-a8c7-53508ce430df/volumes" Jul 12 08:33:38.872342 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:38.872310 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:41.550510 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:41.550470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:41.608050 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:41.608021 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:41.692142 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:41.692109 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:41.790947 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:41.790918 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:41.891933 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:41.891902 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:42.235540 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:33:42.235459 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:08.190045 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:08.190011 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:13.854156 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:13.854125 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:13.914502 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:13.914442 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:14.008395 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:14.008364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:14.108350 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:14.108265 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:29.906913 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:29.906872 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:35.608156 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:35.608122 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:36.711809 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:36.711775 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:46.323967 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:46.323937 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:47.137130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:47.137097 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:52.785120 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:52.785085 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:52.888558 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:52.888524 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:52.982675 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:52.982641 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:53.083099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:53.083068 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:53.814126 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:53.814094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:53.916649 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:53.916614 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:54.013325 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:54.013294 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:55.566314 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:34:55.566281 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:01.386977 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:01.386889 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:01.488132 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:01.488103 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:02.109666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:02.109638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:07.894032 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:07.893999 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:07.969221 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:07.969193 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:08.120662 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.120631 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:08.401616 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.401581 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882"] Jul 12 08:35:08.401900 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.401887 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17762cf-8189-4e73-a8c7-53508ce430df" containerName="istio-proxy" Jul 12 08:35:08.401952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.401903 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17762cf-8189-4e73-a8c7-53508ce430df" containerName="istio-proxy" Jul 12 08:35:08.401987 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.401968 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17762cf-8189-4e73-a8c7-53508ce430df" containerName="istio-proxy" Jul 12 08:35:08.404695 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.404677 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.407123 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.407103 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7937c472-openshift-default-dockercfg-77v8g\"" Jul 12 08:35:08.426327 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.426302 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:08.427238 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.427216 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882"] Jul 12 08:35:08.439651 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439627 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-socket\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.439786 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439666 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-credential-socket\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.439786 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439692 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-data\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.439786 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439767 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-podinfo\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.439952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439867 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-envoy\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.439952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439900 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istiod-ca-cert\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.440063 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.439979 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-token\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.440063 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.440028 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-certs\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.440158 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.440060 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27pnk\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-kube-api-access-27pnk\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.540834 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.540792 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-certs\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541012 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.540845 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27pnk\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-kube-api-access-27pnk\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541012 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.540894 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-socket\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541012 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.540923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-credential-socket\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541012 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.540979 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-data\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541229 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541043 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-podinfo\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541229 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541091 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-envoy\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541229 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541177 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-certs\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541229 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541188 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istiod-ca-cert\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541243 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-credential-socket\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541295 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-token\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541389 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-data\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.541584 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.541468 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-socket\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.542064 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.542040 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istiod-ca-cert\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.543490 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.543468 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-envoy\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.543578 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.543560 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-podinfo\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.548562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.548539 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-token\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.549153 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.549137 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27pnk\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-kube-api-access-27pnk\") pod \"e2e-mt-7937c472-openshift-default-79dd9765c9-78882\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.606979 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.606952 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:08.714442 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.714351 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:08.850416 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.850390 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882"] Jul 12 08:35:08.852690 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:35:08.852660 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae318b42_7a48_4a7a_b662_2e405ca0aa91.slice/crio-f1f5fc15d8323f45087eb53e1fe6984b89dd2687e5b1b8cefbf3343441da0ec1 WatchSource:0}: Error finding container f1f5fc15d8323f45087eb53e1fe6984b89dd2687e5b1b8cefbf3343441da0ec1: Status 404 returned error can't find the container with id f1f5fc15d8323f45087eb53e1fe6984b89dd2687e5b1b8cefbf3343441da0ec1 Jul 12 08:35:08.854802 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.854777 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:08.854869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.854835 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:08.854905 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:08.854867 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:09.073754 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:09.073707 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" event={"ID":"ae318b42-7a48-4a7a-b662-2e405ca0aa91","Type":"ContainerStarted","Data":"fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a"} Jul 12 08:35:09.074113 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:09.073796 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" event={"ID":"ae318b42-7a48-4a7a-b662-2e405ca0aa91","Type":"ContainerStarted","Data":"f1f5fc15d8323f45087eb53e1fe6984b89dd2687e5b1b8cefbf3343441da0ec1"} Jul 12 08:35:09.093589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:09.093541 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" podStartSLOduration=1.093526162 podStartE2EDuration="1.093526162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:35:09.091193529 +0000 UTC m=+1693.536678658" watchObservedRunningTime="2026-07-12 08:35:09.093526162 +0000 UTC m=+1693.539011291" Jul 12 08:35:09.715544 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:09.715509 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:09.720250 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:09.720226 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:10.077725 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:10.077698 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:10.078709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:10.078693 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:11.102832 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:11.102796 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:11.181890 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:11.181853 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:11.250985 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:11.250956 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:11.854704 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:11.854670 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:11.953547 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:11.953508 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:12.027890 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:12.027851 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:15.585784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:15.585730 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:16.200082 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:16.200046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:16.300156 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:16.300117 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:16.625433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:16.625403 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:22.298023 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:22.297925 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:22.397809 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:22.397777 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:22.712385 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:22.712346 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:23.094550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:23.094331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:23.154659 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:23.154592 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:25.771376 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:25.770594 2558 patch_prober.go:28] interesting pod/e2e-mt-7937c472-openshift-default-79dd9765c9-78882 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:60482->10.134.0.31:15021: read: connection reset by peer" start-of-body= Jul 12 08:35:25.771376 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:25.770671 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:60482->10.134.0.31:15021: read: connection reset by peer" Jul 12 08:35:26.148875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:26.148844 2558 generic.go:356] "Generic (PLEG): container finished" podID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerID="fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a" exitCode=0 Jul 12 08:35:26.149045 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:26.148882 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" event={"ID":"ae318b42-7a48-4a7a-b662-2e405ca0aa91","Type":"ContainerDied","Data":"fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a"} Jul 12 08:35:26.149259 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:26.149243 2558 scope.go:117] "RemoveContainer" containerID="fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a" Jul 12 08:35:26.150285 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:26.150253 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:26.150343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:26.150326 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:26.150377 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:26.150359 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:27.154519 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:27.154480 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" event={"ID":"ae318b42-7a48-4a7a-b662-2e405ca0aa91","Type":"ContainerStarted","Data":"fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79"} Jul 12 08:35:27.715554 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:27.715517 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:27.715796 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:27.715776 2558 patch_prober.go:28] interesting pod/e2e-mt-7937c472-openshift-default-79dd9765c9-78882 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 12 08:35:27.715851 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:27.715823 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 12 08:35:28.703919 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:28.703890 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:28.714953 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:28.714928 2558 patch_prober.go:28] interesting pod/e2e-mt-7937c472-openshift-default-79dd9765c9-78882 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 12 08:35:28.715038 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:28.714973 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 12 08:35:28.804997 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:28.804963 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:28.931719 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:28.931678 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882"] Jul 12 08:35:28.950985 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:28.950946 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:29.162897 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:29.162862 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" containerID="cri-o://fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79" gracePeriod=30 Jul 12 08:35:29.297218 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:29.297180 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:29.397715 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:29.397683 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:29.498198 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:29.498109 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:35.129863 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:35.129838 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:35.230698 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:35.230665 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:35.329935 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:35.329900 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:35.473983 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:35.473897 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:36.109342 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.109317 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:36.183641 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183560 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-socket\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.183641 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183597 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27pnk\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-kube-api-access-27pnk\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.183641 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183621 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-certs\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183652 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-token\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183680 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-data\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183707 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-credential-socket\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183766 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-envoy\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183803 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-podinfo\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183826 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istiod-ca-cert\") pod \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\" (UID: \"ae318b42-7a48-4a7a-b662-2e405ca0aa91\") " Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183847 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.183969 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.184186 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.184208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.184208 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.184706 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.184251 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:35:36.184706 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.184108 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:35:36.184706 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.184637 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:35:36.185951 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.185923 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-kube-api-access-27pnk" (OuterVolumeSpecName: "kube-api-access-27pnk") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "kube-api-access-27pnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:35:36.186082 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.186046 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:35:36.186082 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.186067 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:35:36.186260 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.186238 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae318b42-7a48-4a7a-b662-2e405ca0aa91" (UID: "ae318b42-7a48-4a7a-b662-2e405ca0aa91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:35:36.191888 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.191868 2558 generic.go:356] "Generic (PLEG): container finished" podID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerID="fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79" exitCode=0 Jul 12 08:35:36.191953 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.191918 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" Jul 12 08:35:36.192007 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.191915 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" event={"ID":"ae318b42-7a48-4a7a-b662-2e405ca0aa91","Type":"ContainerDied","Data":"fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79"} Jul 12 08:35:36.192103 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.192083 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882" event={"ID":"ae318b42-7a48-4a7a-b662-2e405ca0aa91","Type":"ContainerDied","Data":"f1f5fc15d8323f45087eb53e1fe6984b89dd2687e5b1b8cefbf3343441da0ec1"} Jul 12 08:35:36.192158 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.192130 2558 scope.go:117] "RemoveContainer" containerID="fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79" Jul 12 08:35:36.204152 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.204136 2558 scope.go:117] "RemoveContainer" containerID="fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a" Jul 12 08:35:36.211313 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.211296 2558 scope.go:117] "RemoveContainer" containerID="fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79" Jul 12 08:35:36.211560 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:35:36.211542 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79\": container with ID starting with fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79 not found: ID does not exist" containerID="fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79" Jul 12 08:35:36.211619 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.211566 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79"} err="failed to get container status \"fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79\": rpc error: code = NotFound desc = could not find container \"fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79\": container with ID starting with fc5a8e4e050d74bf9d87554d3404d369a2eab63e351635bdb2b40387ae7aee79 not found: ID does not exist" Jul 12 08:35:36.211619 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.211582 2558 scope.go:117] "RemoveContainer" containerID="fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a" Jul 12 08:35:36.211865 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:35:36.211791 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a\": container with ID starting with fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a not found: ID does not exist" containerID="fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a" Jul 12 08:35:36.211865 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.211810 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a"} err="failed to get container status \"fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a\": rpc error: code = NotFound desc = could not find container \"fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a\": container with ID starting with fbb0eda6f9d33669ba7b2b51ae8fe7bc084c57633766b8af66219a594766ee1a not found: ID does not exist" Jul 12 08:35:36.215516 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.215496 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882"] Jul 12 08:35:36.221291 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.221272 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7937c472-openshift-default-79dd9765c9-78882"] Jul 12 08:35:36.285089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285062 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27pnk\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-kube-api-access-27pnk\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.285089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285086 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.285089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285096 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.285280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285104 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.285280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285112 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.285280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285120 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.285280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.285128 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae318b42-7a48-4a7a-b662-2e405ca0aa91-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:35:36.387243 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:36.387213 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" path="/var/lib/kubelet/pods/ae318b42-7a48-4a7a-b662-2e405ca0aa91/volumes" Jul 12 08:35:41.222228 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:41.222183 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:41.322238 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:41.322208 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:48.026095 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.026048 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:48.313723 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.313693 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l"] Jul 12 08:35:48.314089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.314073 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" Jul 12 08:35:48.314145 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.314092 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" Jul 12 08:35:48.314178 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.314167 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" Jul 12 08:35:48.318421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.318405 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.320803 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.320782 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a5ecc4df-openshift-default-dockercfg-lvrg2\"" Jul 12 08:35:48.328464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.328438 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l"] Jul 12 08:35:48.336714 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.336693 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:48.380116 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-data\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380126 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-socket\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380153 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-envoy\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-certs\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380195 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-podinfo\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380364 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-credential-socket\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380364 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380310 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-token\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380427 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380378 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae2b126-2de1-4194-8528-a5303d6976f0-istiod-ca-cert\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.380462 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.380425 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdcr7\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-kube-api-access-kdcr7\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481325 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481288 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae2b126-2de1-4194-8528-a5303d6976f0-istiod-ca-cert\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481516 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481339 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdcr7\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-kube-api-access-kdcr7\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-data\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481553 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-socket\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481589 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481579 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-envoy\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-certs\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-podinfo\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-credential-socket\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.481781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481709 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-token\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.482060 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481948 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-socket\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.482060 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.481991 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-data\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.482060 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.482040 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-certs\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.482224 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.482160 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-credential-socket\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.482224 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.482211 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae2b126-2de1-4194-8528-a5303d6976f0-istiod-ca-cert\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.483859 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.483837 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-envoy\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.484208 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.484190 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-podinfo\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.489668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.489648 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-token\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.489790 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.489680 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdcr7\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-kube-api-access-kdcr7\") pod \"e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.491168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.491143 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:48.629698 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.629610 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:48.756243 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.756215 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l"] Jul 12 08:35:48.760684 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:35:48.760657 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae2b126_2de1_4194_8528_a5303d6976f0.slice/crio-98c75ce4361c19032d97c74083c950a26491ae2b085444755bc0a8dbf7f5462c WatchSource:0}: Error finding container 98c75ce4361c19032d97c74083c950a26491ae2b085444755bc0a8dbf7f5462c: Status 404 returned error can't find the container with id 98c75ce4361c19032d97c74083c950a26491ae2b085444755bc0a8dbf7f5462c Jul 12 08:35:48.762617 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.762586 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:48.762705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.762653 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:48.762705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:48.762680 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:35:49.248245 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:49.248207 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" event={"ID":"7ae2b126-2de1-4194-8528-a5303d6976f0","Type":"ContainerStarted","Data":"bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989"} Jul 12 08:35:49.248691 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:49.248267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" event={"ID":"7ae2b126-2de1-4194-8528-a5303d6976f0","Type":"ContainerStarted","Data":"98c75ce4361c19032d97c74083c950a26491ae2b085444755bc0a8dbf7f5462c"} Jul 12 08:35:49.266911 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:49.266862 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podStartSLOduration=1.266846597 podStartE2EDuration="1.266846597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:35:49.265290693 +0000 UTC m=+1733.710775845" watchObservedRunningTime="2026-07-12 08:35:49.266846597 +0000 UTC m=+1733.712331726" Jul 12 08:35:49.630245 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:49.630213 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:49.634862 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:49.634835 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:50.252942 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:50.252911 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:50.253922 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:50.253899 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:35:51.148172 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:51.148073 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.231138 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:51.231084 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.294945 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:51.294911 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.424345 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:51.424257 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.804057 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:51.803970 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.906986 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:51.906950 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:55.648086 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:35:55.648054 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:05.254079 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.253983 2558 patch_prober.go:28] interesting pod/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 12 08:36:05.254079 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.254055 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 12 08:36:05.320061 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.320025 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerID="bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989" exitCode=0 Jul 12 08:36:05.320232 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.320094 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" event={"ID":"7ae2b126-2de1-4194-8528-a5303d6976f0","Type":"ContainerDied","Data":"bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989"} Jul 12 08:36:05.320468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.320455 2558 scope.go:117] "RemoveContainer" containerID="bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989" Jul 12 08:36:05.321491 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.321464 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:36:05.321591 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.321549 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:36:05.321651 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.321596 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:36:05.379903 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.379877 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6"] Jul 12 08:36:05.380195 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.380180 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" Jul 12 08:36:05.380278 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.380203 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" Jul 12 08:36:05.380334 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.380279 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae318b42-7a48-4a7a-b662-2e405ca0aa91" containerName="istio-proxy" Jul 12 08:36:05.383876 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.383852 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.386564 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.386544 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a5ecc4df\"/\"kube-root-ca.crt\"" Jul 12 08:36:05.386564 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.386557 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a5ecc4df\"/\"openshift-service-ca.crt\"" Jul 12 08:36:05.386725 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.386659 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a5ecc4df\"/\"e2e-lifecycle-model-a5ecc4df-kserve-self-signed-certs\"" Jul 12 08:36:05.386807 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.386719 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a5ecc4df\"/\"default-dockercfg-b5f9p\"" Jul 12 08:36:05.392116 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.392062 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6"] Jul 12 08:36:05.407632 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.407604 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:05.423717 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.423689 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-tmp-dir\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.423849 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.423736 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/660569b8-1086-420c-8de3-4e886530e3bd-tls-certs\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.423849 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.423803 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-home\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.423959 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.423923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-model-cache\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.424042 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.424011 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-dshm\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.424101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.424087 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clw7b\" (UniqueName: \"kubernetes.io/projected/660569b8-1086-420c-8de3-4e886530e3bd-kube-api-access-clw7b\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525321 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-dshm\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525321 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525286 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clw7b\" (UniqueName: \"kubernetes.io/projected/660569b8-1086-420c-8de3-4e886530e3bd-kube-api-access-clw7b\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-tmp-dir\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/660569b8-1086-420c-8de3-4e886530e3bd-tls-certs\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-home\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525802 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525571 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-model-cache\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525802 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525719 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-tmp-dir\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525899 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-model-cache\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.525939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.525915 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-home\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.527564 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.527537 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-dshm\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.528163 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.528140 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/660569b8-1086-420c-8de3-4e886530e3bd-tls-certs\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.532857 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.532837 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clw7b\" (UniqueName: \"kubernetes.io/projected/660569b8-1086-420c-8de3-4e886530e3bd-kube-api-access-clw7b\") pod \"e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.694269 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.694230 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:05.835840 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:05.835810 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6"] Jul 12 08:36:05.838621 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:36:05.838597 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod660569b8_1086_420c_8de3_4e886530e3bd.slice/crio-5e1d77370409391186323a027d1e6817fc56a3670c38015a70eed11451a4c285 WatchSource:0}: Error finding container 5e1d77370409391186323a027d1e6817fc56a3670c38015a70eed11451a4c285: Status 404 returned error can't find the container with id 5e1d77370409391186323a027d1e6817fc56a3670c38015a70eed11451a4c285 Jul 12 08:36:06.329294 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.328372 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" event={"ID":"7ae2b126-2de1-4194-8528-a5303d6976f0","Type":"ContainerStarted","Data":"9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db"} Jul 12 08:36:06.332037 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.331474 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" event={"ID":"660569b8-1086-420c-8de3-4e886530e3bd","Type":"ContainerStarted","Data":"06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1"} Jul 12 08:36:06.332037 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.331518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" event={"ID":"660569b8-1086-420c-8de3-4e886530e3bd","Type":"ContainerStarted","Data":"5e1d77370409391186323a027d1e6817fc56a3670c38015a70eed11451a4c285"} Jul 12 08:36:06.365261 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.365206 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" podStartSLOduration=1.365190132 podStartE2EDuration="1.365190132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:36:06.364048715 +0000 UTC m=+1750.809533847" watchObservedRunningTime="2026-07-12 08:36:06.365190132 +0000 UTC m=+1750.810675263" Jul 12 08:36:06.631330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.630824 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:36:06.631330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.630952 2558 patch_prober.go:28] interesting pod/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 12 08:36:06.631330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:06.631001 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 12 08:36:07.630332 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:07.630295 2558 patch_prober.go:28] interesting pod/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 12 08:36:07.630931 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:07.630896 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 12 08:36:08.630410 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:08.630368 2558 patch_prober.go:28] interesting pod/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 12 08:36:08.630986 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:08.630952 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 12 08:36:09.630722 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:09.630683 2558 patch_prober.go:28] interesting pod/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 12 08:36:09.631303 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:09.631269 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 12 08:36:10.634320 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:10.634287 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:36:10.634803 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:10.634677 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:36:10.635474 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:10.635458 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:36:15.694485 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:15.694451 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:15.694880 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:15.694522 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:15.706476 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:15.706453 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:16.395200 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:16.395171 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:23.004292 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:23.004260 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:23.355620 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:23.355558 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:26.572897 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:26.572866 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:26.673580 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:26.673547 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:29.018623 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:29.018593 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:33.998867 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:33.998833 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6"] Jul 12 08:36:34.094131 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:34.094096 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:37.931585 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:37.931544 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:49.018929 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.018836 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" podUID="660569b8-1086-420c-8de3-4e886530e3bd" containerName="main" containerID="cri-o://06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1" gracePeriod=45 Jul 12 08:36:49.267303 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.267281 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:49.321541 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321509 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-model-cache\") pod \"660569b8-1086-420c-8de3-4e886530e3bd\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " Jul 12 08:36:49.321705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321547 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clw7b\" (UniqueName: \"kubernetes.io/projected/660569b8-1086-420c-8de3-4e886530e3bd-kube-api-access-clw7b\") pod \"660569b8-1086-420c-8de3-4e886530e3bd\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " Jul 12 08:36:49.321705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321602 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-tmp-dir\") pod \"660569b8-1086-420c-8de3-4e886530e3bd\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " Jul 12 08:36:49.321705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321662 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-dshm\") pod \"660569b8-1086-420c-8de3-4e886530e3bd\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " Jul 12 08:36:49.321705 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321701 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/660569b8-1086-420c-8de3-4e886530e3bd-tls-certs\") pod \"660569b8-1086-420c-8de3-4e886530e3bd\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " Jul 12 08:36:49.322009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321726 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-home\") pod \"660569b8-1086-420c-8de3-4e886530e3bd\" (UID: \"660569b8-1086-420c-8de3-4e886530e3bd\") " Jul 12 08:36:49.322009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321851 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "660569b8-1086-420c-8de3-4e886530e3bd" (UID: "660569b8-1086-420c-8de3-4e886530e3bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:36:49.322009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.321923 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "660569b8-1086-420c-8de3-4e886530e3bd" (UID: "660569b8-1086-420c-8de3-4e886530e3bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:36:49.322171 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.322040 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-model-cache\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:36:49.322171 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.322061 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-tmp-dir\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:36:49.322171 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.322123 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-home" (OuterVolumeSpecName: "home") pod "660569b8-1086-420c-8de3-4e886530e3bd" (UID: "660569b8-1086-420c-8de3-4e886530e3bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:36:49.324384 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.324353 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/660569b8-1086-420c-8de3-4e886530e3bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "660569b8-1086-420c-8de3-4e886530e3bd" (UID: "660569b8-1086-420c-8de3-4e886530e3bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:36:49.324384 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.324373 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/660569b8-1086-420c-8de3-4e886530e3bd-kube-api-access-clw7b" (OuterVolumeSpecName: "kube-api-access-clw7b") pod "660569b8-1086-420c-8de3-4e886530e3bd" (UID: "660569b8-1086-420c-8de3-4e886530e3bd"). InnerVolumeSpecName "kube-api-access-clw7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:36:49.324384 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.324357 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-dshm" (OuterVolumeSpecName: "dshm") pod "660569b8-1086-420c-8de3-4e886530e3bd" (UID: "660569b8-1086-420c-8de3-4e886530e3bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:36:49.423129 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.423091 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clw7b\" (UniqueName: \"kubernetes.io/projected/660569b8-1086-420c-8de3-4e886530e3bd-kube-api-access-clw7b\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:36:49.423129 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.423123 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-dshm\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:36:49.423322 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.423137 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/660569b8-1086-420c-8de3-4e886530e3bd-tls-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:36:49.423322 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.423149 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/660569b8-1086-420c-8de3-4e886530e3bd-home\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:36:49.517461 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.517430 2558 generic.go:356] "Generic (PLEG): container finished" podID="660569b8-1086-420c-8de3-4e886530e3bd" containerID="06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1" exitCode=0 Jul 12 08:36:49.517615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.517471 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" event={"ID":"660569b8-1086-420c-8de3-4e886530e3bd","Type":"ContainerDied","Data":"06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1"} Jul 12 08:36:49.517615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.517500 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" event={"ID":"660569b8-1086-420c-8de3-4e886530e3bd","Type":"ContainerDied","Data":"5e1d77370409391186323a027d1e6817fc56a3670c38015a70eed11451a4c285"} Jul 12 08:36:49.517615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.517520 2558 scope.go:117] "RemoveContainer" containerID="06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1" Jul 12 08:36:49.517615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.517473 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6" Jul 12 08:36:49.526468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.526451 2558 scope.go:117] "RemoveContainer" containerID="06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1" Jul 12 08:36:49.526769 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:36:49.526725 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1\": container with ID starting with 06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1 not found: ID does not exist" containerID="06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1" Jul 12 08:36:49.526831 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.526772 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1"} err="failed to get container status \"06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1\": rpc error: code = NotFound desc = could not find container \"06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1\": container with ID starting with 06009df684d5db844bac3b1a3f33d59052e13f8cc0c6ad87e9bf6a78897221b1 not found: ID does not exist" Jul 12 08:36:49.539348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.539324 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6"] Jul 12 08:36:49.545086 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:49.545064 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a5ecc4df/e2e-lifecycle-model-a5ecc4df-kserve-b9588974c-9lww6"] Jul 12 08:36:50.386943 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:50.386913 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="660569b8-1086-420c-8de3-4e886530e3bd" path="/var/lib/kubelet/pods/660569b8-1086-420c-8de3-4e886530e3bd/volumes" Jul 12 08:36:56.203708 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:56.203677 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:36:56.207363 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:36:56.207343 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:37:01.642199 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:01.642154 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:10.307150 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:10.307118 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:11.206763 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:11.206712 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:16.755129 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:16.755093 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:16.814762 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:16.814713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:16.999737 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:16.999705 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l"] Jul 12 08:37:17.000007 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:16.999969 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" containerID="cri-o://9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db" gracePeriod=30 Jul 12 08:37:17.042729 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:17.042650 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:18.284681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:18.284647 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:18.396791 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:18.396736 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:18.575722 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:18.575697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:19.227115 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:19.227082 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:19.326692 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:19.326662 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:19.426758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:19.426716 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:20.731218 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:20.731180 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:22.258464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.258435 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:37:22.299822 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299736 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-data\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.299822 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299797 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-token\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299825 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-envoy\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299852 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae2b126-2de1-4194-8528-a5303d6976f0-istiod-ca-cert\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299893 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-podinfo\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299924 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-socket\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.299959 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-credential-socket\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300019 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.300006 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdcr7\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-kube-api-access-kdcr7\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.300347 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.300317 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:37:22.300400 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.300331 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ae2b126-2de1-4194-8528-a5303d6976f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:37:22.300511 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.300477 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:37:22.300653 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.300628 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:37:22.302202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.302168 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:37:22.302202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.302179 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:37:22.302318 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.302212 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:37:22.302427 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.302409 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-kube-api-access-kdcr7" (OuterVolumeSpecName: "kube-api-access-kdcr7") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "kube-api-access-kdcr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:37:22.400601 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400575 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-certs\") pod \"7ae2b126-2de1-4194-8528-a5303d6976f0\" (UID: \"7ae2b126-2de1-4194-8528-a5303d6976f0\") " Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400691 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae2b126-2de1-4194-8528-a5303d6976f0-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400702 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400710 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400718 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400727 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdcr7\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-kube-api-access-kdcr7\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400734 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400758 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400759 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400984 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400767 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.400984 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.400811 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ae2b126-2de1-4194-8528-a5303d6976f0" (UID: "7ae2b126-2de1-4194-8528-a5303d6976f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:37:22.501076 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.501034 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae2b126-2de1-4194-8528-a5303d6976f0-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:37:22.653523 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.653486 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerID="9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db" exitCode=0 Jul 12 08:37:22.653710 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.653542 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" Jul 12 08:37:22.653710 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.653576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" event={"ID":"7ae2b126-2de1-4194-8528-a5303d6976f0","Type":"ContainerDied","Data":"9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db"} Jul 12 08:37:22.653710 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.653642 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l" event={"ID":"7ae2b126-2de1-4194-8528-a5303d6976f0","Type":"ContainerDied","Data":"98c75ce4361c19032d97c74083c950a26491ae2b085444755bc0a8dbf7f5462c"} Jul 12 08:37:22.653710 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.653673 2558 scope.go:117] "RemoveContainer" containerID="9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db" Jul 12 08:37:22.663584 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.663560 2558 scope.go:117] "RemoveContainer" containerID="bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989" Jul 12 08:37:22.671969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.671945 2558 scope.go:117] "RemoveContainer" containerID="9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db" Jul 12 08:37:22.672205 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:37:22.672183 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db\": container with ID starting with 9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db not found: ID does not exist" containerID="9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db" Jul 12 08:37:22.672272 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.672213 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db"} err="failed to get container status \"9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db\": rpc error: code = NotFound desc = could not find container \"9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db\": container with ID starting with 9451722ff7ac967e9ecebc4b3f739004ac55f5a5bfad74eeb7c31ddc8f64d0db not found: ID does not exist" Jul 12 08:37:22.672272 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.672234 2558 scope.go:117] "RemoveContainer" containerID="bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989" Jul 12 08:37:22.672463 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:37:22.672445 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989\": container with ID starting with bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989 not found: ID does not exist" containerID="bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989" Jul 12 08:37:22.672525 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.672468 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989"} err="failed to get container status \"bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989\": rpc error: code = NotFound desc = could not find container \"bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989\": container with ID starting with bbcdf772c2db2f97e5de0c1aa140a11b521b7e1d5e85c1e487684a73716ab989 not found: ID does not exist" Jul 12 08:37:22.679532 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.678774 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l"] Jul 12 08:37:22.680765 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:22.680723 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a5ecc4df-openshift-default-8cdfb8db7-s5d8l"] Jul 12 08:37:24.387771 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:24.387703 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" path="/var/lib/kubelet/pods/7ae2b126-2de1-4194-8528-a5303d6976f0/volumes" Jul 12 08:37:26.572724 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:26.572693 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:26.680660 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:26.680612 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:27.228413 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:27.228376 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:33.046630 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:33.046593 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:33.110327 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:33.110289 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:33.185571 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:33.185533 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:33.285901 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:33.285871 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:43.726656 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:43.726611 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:37:44.059801 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:37:44.059765 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:05.009073 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:05.009038 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:11.009998 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.009967 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:11.073904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.073854 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:11.347734 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.347700 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:11.638154 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638066 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c"] Jul 12 08:38:11.638451 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638433 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" Jul 12 08:38:11.638537 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638455 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" Jul 12 08:38:11.638537 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638486 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" Jul 12 08:38:11.638537 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638495 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" Jul 12 08:38:11.638537 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638509 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="660569b8-1086-420c-8de3-4e886530e3bd" containerName="main" Jul 12 08:38:11.638731 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638541 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="660569b8-1086-420c-8de3-4e886530e3bd" containerName="main" Jul 12 08:38:11.638731 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638621 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" Jul 12 08:38:11.638731 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638634 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae2b126-2de1-4194-8528-a5303d6976f0" containerName="istio-proxy" Jul 12 08:38:11.638731 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.638647 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="660569b8-1086-420c-8de3-4e886530e3bd" containerName="main" Jul 12 08:38:11.641579 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.641557 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.643834 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.643816 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-050284-openshift-default-dockercfg-8ptns\"" Jul 12 08:38:11.655346 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.655320 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c"] Jul 12 08:38:11.669829 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.669803 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:11.821002 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.820962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-credential-socket\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821002 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821018 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-podinfo\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-envoy\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821070 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-data\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-socket\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821166 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istiod-ca-cert\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821212 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-token\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821251 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821237 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xr4w\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-kube-api-access-5xr4w\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.821465 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.821309 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-certs\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.922922 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.922836 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-certs\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.922922 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.922900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-credential-socket\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.922934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-podinfo\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.922965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-envoy\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.922988 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-data\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923018 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-socket\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923046 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istiod-ca-cert\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923078 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-token\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xr4w\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-kube-api-access-5xr4w\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923525 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923297 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-certs\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923525 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923449 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-socket\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923624 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923584 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-credential-socket\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.923706 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.923685 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-data\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.924052 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.924032 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istiod-ca-cert\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.925609 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.925583 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-podinfo\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.925709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.925587 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-envoy\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.930927 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.930906 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-token\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.931130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.931104 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xr4w\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-kube-api-access-5xr4w\") pod \"e2e-inf-a-050284-openshift-default-669667d99f-cfj8c\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:11.951444 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:11.951423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:12.080765 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.080725 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c"] Jul 12 08:38:12.082909 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:38:12.082872 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ba3421_c2e1_4c27_b4df_7c615e7b8f96.slice/crio-35f378fd09d0a320f142a1b282a60e9c9ee813a6e24559ed2bc105cb217bd4d2 WatchSource:0}: Error finding container 35f378fd09d0a320f142a1b282a60e9c9ee813a6e24559ed2bc105cb217bd4d2: Status 404 returned error can't find the container with id 35f378fd09d0a320f142a1b282a60e9c9ee813a6e24559ed2bc105cb217bd4d2 Jul 12 08:38:12.085072 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.085023 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:12.085162 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.085116 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:12.085202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.085157 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:12.274109 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.273983 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:12.858460 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.858429 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" event={"ID":"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96","Type":"ContainerStarted","Data":"995fd8089233b8a8e4f0f471a8202bbf95cea2d307d6debd8d38cb5fd0051e10"} Jul 12 08:38:12.858631 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.858487 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" event={"ID":"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96","Type":"ContainerStarted","Data":"35f378fd09d0a320f142a1b282a60e9c9ee813a6e24559ed2bc105cb217bd4d2"} Jul 12 08:38:12.880420 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.880372 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podStartSLOduration=1.8803562550000001 podStartE2EDuration="1.880356255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:38:12.878663548 +0000 UTC m=+1877.324148681" watchObservedRunningTime="2026-07-12 08:38:12.880356255 +0000 UTC m=+1877.325841385" Jul 12 08:38:12.952248 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:12.952216 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:13.956682 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:13.956653 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:14.753977 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:14.753943 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:14.848454 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:14.848422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:14.866986 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:14.866962 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:14.867854 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:14.867837 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:14.914519 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:14.914493 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:15.015619 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:15.015528 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:15.332146 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:15.332118 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:15.431552 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:15.431522 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:19.250949 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.250868 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:19.569468 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.569436 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:19.841901 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.841811 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd"] Jul 12 08:38:19.845201 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.845180 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.847478 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.847451 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-34847c-openshift-default-dockercfg-j7bnn\"" Jul 12 08:38:19.857534 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.857513 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd"] Jul 12 08:38:19.869941 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.869914 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:19.894333 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894304 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-credential-socket\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894462 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894338 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-envoy\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894462 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-podinfo\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894462 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894446 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-token\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894493 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-data\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894517 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-socket\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894542 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-certs\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkp4n\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-kube-api-access-tkp4n\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.894633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.894586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istiod-ca-cert\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995133 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995096 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-token\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995133 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995136 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-data\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995358 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-socket\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995358 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-certs\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995469 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995362 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkp4n\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-kube-api-access-tkp4n\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995469 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995427 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istiod-ca-cert\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995570 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995545 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-certs\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995624 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995593 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-data\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995624 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995610 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-credential-socket\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995729 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995618 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-socket\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995729 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995639 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-envoy\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995729 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995672 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-podinfo\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.995931 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.995908 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-credential-socket\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.996137 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.996118 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istiod-ca-cert\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.997925 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.997903 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-envoy\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:19.998065 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:19.998048 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-podinfo\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:20.002099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.002076 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-token\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:20.002631 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.002615 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkp4n\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-kube-api-access-tkp4n\") pod \"e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:20.156583 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.156495 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:20.180061 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.180031 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:20.290602 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.290521 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd"] Jul 12 08:38:20.293006 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:38:20.292967 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b19ed4_20ee_4c61_bea4_ddd0de358a98.slice/crio-2c2ff5f7634cb676b5973da0b15ae3d973e9f63bc9030c2187434c5a22a7c581 WatchSource:0}: Error finding container 2c2ff5f7634cb676b5973da0b15ae3d973e9f63bc9030c2187434c5a22a7c581: Status 404 returned error can't find the container with id 2c2ff5f7634cb676b5973da0b15ae3d973e9f63bc9030c2187434c5a22a7c581 Jul 12 08:38:20.295243 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.295206 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:20.295307 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.295291 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:20.295353 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.295337 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:20.898713 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.898674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" event={"ID":"83b19ed4-20ee-4c61-bea4-ddd0de358a98","Type":"ContainerStarted","Data":"45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f"} Jul 12 08:38:20.898943 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.898725 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" event={"ID":"83b19ed4-20ee-4c61-bea4-ddd0de358a98","Type":"ContainerStarted","Data":"2c2ff5f7634cb676b5973da0b15ae3d973e9f63bc9030c2187434c5a22a7c581"} Jul 12 08:38:20.919154 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:20.919107 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podStartSLOduration=1.919091183 podStartE2EDuration="1.919091183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:38:20.916911353 +0000 UTC m=+1885.362396506" watchObservedRunningTime="2026-07-12 08:38:20.919091183 +0000 UTC m=+1885.364576312" Jul 12 08:38:21.157148 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:21.157064 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:21.161873 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:21.161848 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:21.902719 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:21.902686 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:21.903734 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:21.903709 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:22.794823 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:22.794786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:22.890681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:22.890648 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:22.987140 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:22.987108 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:23.377781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:23.377723 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:23.476280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:23.476247 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:23.574945 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:23.574913 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:27.280802 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:27.280763 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:30.356696 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.355670 2558 patch_prober.go:28] interesting pod/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": read tcp 10.134.0.2:52816->10.134.0.34:15021: read: connection reset by peer" start-of-body= Jul 12 08:38:30.356696 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.355730 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": read tcp 10.134.0.2:52816->10.134.0.34:15021: read: connection reset by peer" Jul 12 08:38:30.950068 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.950032 2558 generic.go:356] "Generic (PLEG): container finished" podID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerID="995fd8089233b8a8e4f0f471a8202bbf95cea2d307d6debd8d38cb5fd0051e10" exitCode=0 Jul 12 08:38:30.950252 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.950078 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" event={"ID":"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96","Type":"ContainerDied","Data":"995fd8089233b8a8e4f0f471a8202bbf95cea2d307d6debd8d38cb5fd0051e10"} Jul 12 08:38:30.950505 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.950491 2558 scope.go:117] "RemoveContainer" containerID="995fd8089233b8a8e4f0f471a8202bbf95cea2d307d6debd8d38cb5fd0051e10" Jul 12 08:38:30.951601 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.951573 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:30.951671 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.951653 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:30.951723 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:30.951707 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:31.955913 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:31.955879 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" event={"ID":"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96","Type":"ContainerStarted","Data":"1e336390d5a6426acdcd05556545db4faaeb04fa16d8ec63ea9a541c9da6413e"} Jul 12 08:38:32.952398 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:32.952352 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:32.952844 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:32.952806 2558 patch_prober.go:28] interesting pod/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 08:38:32.952972 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:32.952900 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 08:38:33.952102 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:33.952065 2558 patch_prober.go:28] interesting pod/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 08:38:33.952646 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:33.952614 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 08:38:34.952306 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:34.952261 2558 patch_prober.go:28] interesting pod/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 08:38:34.952794 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:34.952326 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 08:38:35.579368 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.579331 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg"] Jul 12 08:38:35.583730 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.583713 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.587009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.586984 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-050284\"/\"kube-root-ca.crt\"" Jul 12 08:38:35.587132 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.586990 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-050284\"/\"test-model-050284-kserve-self-signed-certs\"" Jul 12 08:38:35.587132 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.586991 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-050284\"/\"default-dockercfg-4l5zp\"" Jul 12 08:38:35.587132 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.587041 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-050284\"/\"openshift-service-ca.crt\"" Jul 12 08:38:35.596391 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.594227 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg"] Jul 12 08:38:35.640867 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.640809 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-dshm\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.641200 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.641178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-tmp-dir\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.641402 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.641386 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-model-cache\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.641550 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.641536 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-home\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.641781 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.641730 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02fd944c-7652-40cf-9284-b6da0429ce3b-tls-certs\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.642009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.641990 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxns8\" (UniqueName: \"kubernetes.io/projected/02fd944c-7652-40cf-9284-b6da0429ce3b-kube-api-access-cxns8\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.669409 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.669379 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:35.743394 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02fd944c-7652-40cf-9284-b6da0429ce3b-tls-certs\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.743592 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxns8\" (UniqueName: \"kubernetes.io/projected/02fd944c-7652-40cf-9284-b6da0429ce3b-kube-api-access-cxns8\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.743592 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743444 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-dshm\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.743592 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743461 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-tmp-dir\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.743592 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-model-cache\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.743592 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743500 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-home\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.744018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.743993 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-home\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.744191 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.744014 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-model-cache\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.744191 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.744119 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-tmp-dir\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.745854 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.745828 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-dshm\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.746486 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.746469 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02fd944c-7652-40cf-9284-b6da0429ce3b-tls-certs\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.754004 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.753980 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxns8\" (UniqueName: \"kubernetes.io/projected/02fd944c-7652-40cf-9284-b6da0429ce3b-kube-api-access-cxns8\") pod \"test-model-050284-kserve-6f7db45bc-66rhg\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.893734 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.893651 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:35.952281 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.952184 2558 patch_prober.go:28] interesting pod/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 12 08:38:35.952281 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:35.952249 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 12 08:38:36.094023 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.093996 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg"] Jul 12 08:38:36.095910 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:38:36.095883 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02fd944c_7652_40cf_9284_b6da0429ce3b.slice/crio-ba060df09ca4b602b5e88701f0bbed5a2571d317feb24e19d0935006897d3c3b WatchSource:0}: Error finding container ba060df09ca4b602b5e88701f0bbed5a2571d317feb24e19d0935006897d3c3b: Status 404 returned error can't find the container with id ba060df09ca4b602b5e88701f0bbed5a2571d317feb24e19d0935006897d3c3b Jul 12 08:38:36.812677 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.812623 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:36.911708 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.911678 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:36.915615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.915589 2558 patch_prober.go:28] interesting pod/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": read tcp 10.134.0.2:53706->10.134.0.35:15021: read: connection reset by peer" start-of-body= Jul 12 08:38:36.915821 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.915789 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": read tcp 10.134.0.2:53706->10.134.0.35:15021: read: connection reset by peer" Jul 12 08:38:36.983379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.983345 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" event={"ID":"02fd944c-7652-40cf-9284-b6da0429ce3b","Type":"ContainerStarted","Data":"ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38"} Jul 12 08:38:36.983839 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.983394 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" event={"ID":"02fd944c-7652-40cf-9284-b6da0429ce3b","Type":"ContainerStarted","Data":"ba060df09ca4b602b5e88701f0bbed5a2571d317feb24e19d0935006897d3c3b"} Jul 12 08:38:36.984889 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.984863 2558 generic.go:356] "Generic (PLEG): container finished" podID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerID="45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f" exitCode=0 Jul 12 08:38:36.985006 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.984918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" event={"ID":"83b19ed4-20ee-4c61-bea4-ddd0de358a98","Type":"ContainerDied","Data":"45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f"} Jul 12 08:38:36.985234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.985221 2558 scope.go:117] "RemoveContainer" containerID="45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f" Jul 12 08:38:36.986181 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.986153 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:36.986288 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.986227 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:36.986288 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.986264 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:38:36.993975 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.993953 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:36.994357 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.994325 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:36.996071 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:36.996050 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:38:37.000412 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:37.000373 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" podStartSLOduration=2.000361458 podStartE2EDuration="2.000361458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:38:36.998881274 +0000 UTC m=+1901.444366405" watchObservedRunningTime="2026-07-12 08:38:37.000361458 +0000 UTC m=+1901.445846588" Jul 12 08:38:37.992262 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:37.992223 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" event={"ID":"83b19ed4-20ee-4c61-bea4-ddd0de358a98","Type":"ContainerStarted","Data":"aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e"} Jul 12 08:38:38.157698 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:38.157657 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:38.157954 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:38.157929 2558 patch_prober.go:28] interesting pod/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 08:38:38.158041 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:38.157984 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 08:38:39.157128 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:39.157094 2558 patch_prober.go:28] interesting pod/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 08:38:39.157517 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:39.157158 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 08:38:40.157305 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:40.157267 2558 patch_prober.go:28] interesting pod/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 08:38:40.157687 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:40.157353 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 08:38:41.157319 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:41.157284 2558 patch_prober.go:28] interesting pod/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 12 08:38:41.157694 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:41.157355 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 12 08:38:42.160488 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:42.160458 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:42.160869 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:42.160706 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:42.161402 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:42.161385 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:38:44.210354 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:44.210318 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:44.310791 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:44.310763 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:45.894479 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:45.894445 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:45.894479 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:45.894488 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:45.906623 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:45.906599 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:38:46.042173 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:38:46.042144 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:39:05.910874 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:05.910841 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:25.564317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:25.564282 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg"] Jul 12 08:39:25.574597 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:25.574562 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:26.489950 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:26.489917 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:26.700083 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:26.700050 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:27.148163 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:27.148132 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:27.248387 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:27.248351 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:27.349976 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:27.349941 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:27.468570 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:27.468473 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:32.226148 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:32.226108 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:39:40.584405 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.584358 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" podUID="02fd944c-7652-40cf-9284-b6da0429ce3b" containerName="main" containerID="cri-o://ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38" gracePeriod=45 Jul 12 08:39:40.825353 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.825326 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:39:40.904914 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.904818 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-dshm\") pod \"02fd944c-7652-40cf-9284-b6da0429ce3b\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " Jul 12 08:39:40.904914 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.904875 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02fd944c-7652-40cf-9284-b6da0429ce3b-tls-certs\") pod \"02fd944c-7652-40cf-9284-b6da0429ce3b\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " Jul 12 08:39:40.904914 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.904914 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-tmp-dir\") pod \"02fd944c-7652-40cf-9284-b6da0429ce3b\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " Jul 12 08:39:40.905195 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.904935 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-model-cache\") pod \"02fd944c-7652-40cf-9284-b6da0429ce3b\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " Jul 12 08:39:40.905195 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.904966 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-home\") pod \"02fd944c-7652-40cf-9284-b6da0429ce3b\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " Jul 12 08:39:40.905195 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.904985 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxns8\" (UniqueName: \"kubernetes.io/projected/02fd944c-7652-40cf-9284-b6da0429ce3b-kube-api-access-cxns8\") pod \"02fd944c-7652-40cf-9284-b6da0429ce3b\" (UID: \"02fd944c-7652-40cf-9284-b6da0429ce3b\") " Jul 12 08:39:40.905341 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.905236 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02fd944c-7652-40cf-9284-b6da0429ce3b" (UID: "02fd944c-7652-40cf-9284-b6da0429ce3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:39:40.905341 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.905248 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-model-cache" (OuterVolumeSpecName: "model-cache") pod "02fd944c-7652-40cf-9284-b6da0429ce3b" (UID: "02fd944c-7652-40cf-9284-b6da0429ce3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:39:40.905341 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.905259 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-home" (OuterVolumeSpecName: "home") pod "02fd944c-7652-40cf-9284-b6da0429ce3b" (UID: "02fd944c-7652-40cf-9284-b6da0429ce3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:39:40.907175 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.907151 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-dshm" (OuterVolumeSpecName: "dshm") pod "02fd944c-7652-40cf-9284-b6da0429ce3b" (UID: "02fd944c-7652-40cf-9284-b6da0429ce3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:39:40.907264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.907241 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02fd944c-7652-40cf-9284-b6da0429ce3b-kube-api-access-cxns8" (OuterVolumeSpecName: "kube-api-access-cxns8") pod "02fd944c-7652-40cf-9284-b6da0429ce3b" (UID: "02fd944c-7652-40cf-9284-b6da0429ce3b"). InnerVolumeSpecName "kube-api-access-cxns8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:39:40.907347 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:40.907261 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02fd944c-7652-40cf-9284-b6da0429ce3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02fd944c-7652-40cf-9284-b6da0429ce3b" (UID: "02fd944c-7652-40cf-9284-b6da0429ce3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:39:41.005969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.005934 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-tmp-dir\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:39:41.005969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.005962 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-model-cache\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:39:41.005969 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.005976 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-home\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:39:41.006205 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.005988 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxns8\" (UniqueName: \"kubernetes.io/projected/02fd944c-7652-40cf-9284-b6da0429ce3b-kube-api-access-cxns8\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:39:41.006205 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.005998 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02fd944c-7652-40cf-9284-b6da0429ce3b-dshm\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:39:41.006205 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.006009 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02fd944c-7652-40cf-9284-b6da0429ce3b-tls-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:39:41.261886 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.261794 2558 generic.go:356] "Generic (PLEG): container finished" podID="02fd944c-7652-40cf-9284-b6da0429ce3b" containerID="ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38" exitCode=0 Jul 12 08:39:41.261886 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.261835 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" Jul 12 08:39:41.261886 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.261855 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" event={"ID":"02fd944c-7652-40cf-9284-b6da0429ce3b","Type":"ContainerDied","Data":"ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38"} Jul 12 08:39:41.262128 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.261893 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg" event={"ID":"02fd944c-7652-40cf-9284-b6da0429ce3b","Type":"ContainerDied","Data":"ba060df09ca4b602b5e88701f0bbed5a2571d317feb24e19d0935006897d3c3b"} Jul 12 08:39:41.262128 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.261918 2558 scope.go:117] "RemoveContainer" containerID="ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38" Jul 12 08:39:41.270554 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.270535 2558 scope.go:117] "RemoveContainer" containerID="ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38" Jul 12 08:39:41.270867 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:39:41.270848 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38\": container with ID starting with ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38 not found: ID does not exist" containerID="ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38" Jul 12 08:39:41.270934 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.270876 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38"} err="failed to get container status \"ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38\": rpc error: code = NotFound desc = could not find container \"ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38\": container with ID starting with ff17e110c95aff5556f9618acbdf1aad09f56d5fb4aedde2048afb95eac98b38 not found: ID does not exist" Jul 12 08:39:41.284080 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.282674 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg"] Jul 12 08:39:41.287882 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:41.287852 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-050284/test-model-050284-kserve-6f7db45bc-66rhg"] Jul 12 08:39:42.387436 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:42.387399 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02fd944c-7652-40cf-9284-b6da0429ce3b" path="/var/lib/kubelet/pods/02fd944c-7652-40cf-9284-b6da0429ce3b/volumes" Jul 12 08:39:45.926034 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:39:45.925997 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:09.021763 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:09.021713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:09.255846 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:09.255809 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c"] Jul 12 08:40:09.256101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:09.256046 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" containerID="cri-o://1e336390d5a6426acdcd05556545db4faaeb04fa16d8ec63ea9a541c9da6413e" gracePeriod=30 Jul 12 08:40:09.277159 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:09.277082 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:09.650010 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:09.649964 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:10.058235 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:10.058210 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:10.166755 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:10.166722 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:10.315810 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:10.315710 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:10.373674 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:10.373638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:14.401875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.401301 2558 generic.go:356] "Generic (PLEG): container finished" podID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerID="1e336390d5a6426acdcd05556545db4faaeb04fa16d8ec63ea9a541c9da6413e" exitCode=0 Jul 12 08:40:14.401875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.401471 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" event={"ID":"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96","Type":"ContainerDied","Data":"1e336390d5a6426acdcd05556545db4faaeb04fa16d8ec63ea9a541c9da6413e"} Jul 12 08:40:14.401875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.401545 2558 scope.go:117] "RemoveContainer" containerID="995fd8089233b8a8e4f0f471a8202bbf95cea2d307d6debd8d38cb5fd0051e10" Jul 12 08:40:14.510464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.510436 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:40:14.591280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591251 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istiod-ca-cert\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591318 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-credential-socket\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591349 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-envoy\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591368 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-socket\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591397 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xr4w\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-kube-api-access-5xr4w\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591426 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-podinfo\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591464 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591448 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-certs\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591856 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591486 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-token\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591856 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591526 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-data\") pod \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\" (UID: \"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96\") " Jul 12 08:40:14.591856 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591640 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:40:14.591856 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591732 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:14.592062 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591868 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:14.592062 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591897 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:14.592062 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591906 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.592062 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.591964 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.592437 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.592414 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:14.593752 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.593708 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:14.593887 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.593865 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:40:14.594084 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.594062 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:40:14.594160 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.594073 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-kube-api-access-5xr4w" (OuterVolumeSpecName: "kube-api-access-5xr4w") pod "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" (UID: "e9ba3421-c2e1-4c27-b4df-7c615e7b8f96"). InnerVolumeSpecName "kube-api-access-5xr4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:40:14.692614 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692575 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.692614 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692609 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.692878 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692622 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xr4w\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-kube-api-access-5xr4w\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.692878 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692653 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.692878 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692665 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.692878 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692675 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:14.692878 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:14.692686 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:15.406558 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.406527 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" Jul 12 08:40:15.407032 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.406570 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c" event={"ID":"e9ba3421-c2e1-4c27-b4df-7c615e7b8f96","Type":"ContainerDied","Data":"35f378fd09d0a320f142a1b282a60e9c9ee813a6e24559ed2bc105cb217bd4d2"} Jul 12 08:40:15.407032 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.406627 2558 scope.go:117] "RemoveContainer" containerID="1e336390d5a6426acdcd05556545db4faaeb04fa16d8ec63ea9a541c9da6413e" Jul 12 08:40:15.427437 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.427413 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c"] Jul 12 08:40:15.432787 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.432763 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-050284-openshift-default-669667d99f-cfj8c"] Jul 12 08:40:15.809627 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.809596 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:15.910009 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:15.909980 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:16.038451 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.038420 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd"] Jul 12 08:40:16.038669 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.038629 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" containerID="cri-o://aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e" gracePeriod=30 Jul 12 08:40:16.060809 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.060721 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:16.387849 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.387767 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" path="/var/lib/kubelet/pods/e9ba3421-c2e1-4c27-b4df-7c615e7b8f96/volumes" Jul 12 08:40:16.619112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.619084 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:16.937333 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937296 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627"] Jul 12 08:40:16.937753 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937728 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" Jul 12 08:40:16.937805 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937762 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" Jul 12 08:40:16.937805 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937783 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02fd944c-7652-40cf-9284-b6da0429ce3b" containerName="main" Jul 12 08:40:16.937805 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937793 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02fd944c-7652-40cf-9284-b6da0429ce3b" containerName="main" Jul 12 08:40:16.937898 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937864 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" Jul 12 08:40:16.937898 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.937874 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02fd944c-7652-40cf-9284-b6da0429ce3b" containerName="main" Jul 12 08:40:16.942309 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.942289 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:16.944699 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.944673 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-449923-openshift-default-dockercfg-jm5k4\"" Jul 12 08:40:16.951686 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.951658 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627"] Jul 12 08:40:16.961554 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:16.961532 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:17.011883 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.011850 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-envoy\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.011883 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.011887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af6fa82-e51f-49b4-8815-f75447058db2-istio-podinfo\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.011909 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-credential-socket\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.011928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-data\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.011967 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-socket\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.011993 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af6fa82-e51f-49b4-8815-f75447058db2-istiod-ca-cert\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012089 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.012068 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-istio-token\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012248 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.012143 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-certs\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.012248 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.012168 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkkk\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-kube-api-access-vjkkk\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113245 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-socket\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113308 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af6fa82-e51f-49b4-8815-f75447058db2-istiod-ca-cert\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-istio-token\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113370 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-certs\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113388 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkkk\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-kube-api-access-vjkkk\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113423 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-envoy\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113459 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af6fa82-e51f-49b4-8815-f75447058db2-istio-podinfo\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113488 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-credential-socket\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113572 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113511 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-data\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113975 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113711 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-socket\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113975 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113800 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-certs\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.113975 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113838 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-data\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.114136 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.113986 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-credential-socket\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.114136 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.114106 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af6fa82-e51f-49b4-8815-f75447058db2-istiod-ca-cert\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.115881 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.115860 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-envoy\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.116132 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.116100 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af6fa82-e51f-49b4-8815-f75447058db2-istio-podinfo\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.121921 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.121885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-istio-token\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.122326 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.122308 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkkk\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-kube-api-access-vjkkk\") pod \"e2e-api-a-449923-openshift-default-7d96dc75b9-b2627\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.212061 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.211980 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:17.253482 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.253434 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:17.384033 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.384010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627"] Jul 12 08:40:17.386129 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:40:17.386104 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af6fa82_e51f_49b4_8815_f75447058db2.slice/crio-d2a1c23f98e225e8c3aec50fa20a37816d0c4369af52fd4e4b2edb0eaca85291 WatchSource:0}: Error finding container d2a1c23f98e225e8c3aec50fa20a37816d0c4369af52fd4e4b2edb0eaca85291: Status 404 returned error can't find the container with id d2a1c23f98e225e8c3aec50fa20a37816d0c4369af52fd4e4b2edb0eaca85291 Jul 12 08:40:17.388180 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.388150 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:17.388264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.388210 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:17.388264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.388240 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:17.416657 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:17.416628 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" event={"ID":"0af6fa82-e51f-49b4-8815-f75447058db2","Type":"ContainerStarted","Data":"d2a1c23f98e225e8c3aec50fa20a37816d0c4369af52fd4e4b2edb0eaca85291"} Jul 12 08:40:18.421616 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:18.421575 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" event={"ID":"0af6fa82-e51f-49b4-8815-f75447058db2","Type":"ContainerStarted","Data":"c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c"} Jul 12 08:40:18.442582 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:18.442469 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podStartSLOduration=2.4424491440000002 podStartE2EDuration="2.442449144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:40:18.441372754 +0000 UTC m=+2002.886857897" watchObservedRunningTime="2026-07-12 08:40:18.442449144 +0000 UTC m=+2002.887934277" Jul 12 08:40:19.254355 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:19.254317 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:19.259123 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:19.259095 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:19.426157 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:19.426126 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:19.427185 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:19.427165 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:20.405657 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:20.405624 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:20.496924 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:20.496895 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:20.592073 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:20.592036 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:20.908614 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:20.908584 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:21.152613 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.152573 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:21.255247 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.255220 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:21.290956 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.290931 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:40:21.348507 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348475 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-credential-socket\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348521 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-certs\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348545 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-data\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348565 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-socket\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348591 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-podinfo\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348608 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istiod-ca-cert\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348638 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-envoy\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.348681 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348679 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-token\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348715 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkp4n\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-kube-api-access-tkp4n\") pod \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\" (UID: \"83b19ed4-20ee-4c61-bea4-ddd0de358a98\") " Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348779 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348873 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.348915 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.349056 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.349078 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.349101 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.349098 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.349467 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.349421 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:40:21.349467 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.349449 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-data" (OuterVolumeSpecName: "istio-data") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:21.350889 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.350862 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-token" (OuterVolumeSpecName: "istio-token") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:40:21.351023 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.351002 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:40:21.351245 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.351219 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-kube-api-access-tkp4n" (OuterVolumeSpecName: "kube-api-access-tkp4n") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "kube-api-access-tkp4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:40:21.351348 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.351244 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83b19ed4-20ee-4c61-bea4-ddd0de358a98" (UID: "83b19ed4-20ee-4c61-bea4-ddd0de358a98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:21.436108 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.436078 2558 generic.go:356] "Generic (PLEG): container finished" podID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerID="aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e" exitCode=0 Jul 12 08:40:21.436291 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.436128 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" Jul 12 08:40:21.436291 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.436175 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" event={"ID":"83b19ed4-20ee-4c61-bea4-ddd0de358a98","Type":"ContainerDied","Data":"aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e"} Jul 12 08:40:21.436291 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.436237 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd" event={"ID":"83b19ed4-20ee-4c61-bea4-ddd0de358a98","Type":"ContainerDied","Data":"2c2ff5f7634cb676b5973da0b15ae3d973e9f63bc9030c2187434c5a22a7c581"} Jul 12 08:40:21.436291 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.436260 2558 scope.go:117] "RemoveContainer" containerID="aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e" Jul 12 08:40:21.445247 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.445231 2558 scope.go:117] "RemoveContainer" containerID="45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f" Jul 12 08:40:21.450456 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.450433 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.450456 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.450457 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.450615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.450470 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.450615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.450483 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.450615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.450494 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.450615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.450506 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkp4n\" (UniqueName: \"kubernetes.io/projected/83b19ed4-20ee-4c61-bea4-ddd0de358a98-kube-api-access-tkp4n\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:21.453060 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.453045 2558 scope.go:117] "RemoveContainer" containerID="aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e" Jul 12 08:40:21.453304 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:40:21.453286 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e\": container with ID starting with aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e not found: ID does not exist" containerID="aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e" Jul 12 08:40:21.453346 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.453316 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e"} err="failed to get container status \"aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e\": rpc error: code = NotFound desc = could not find container \"aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e\": container with ID starting with aacad990e272ddf4d603609aec252bfb0cd0c7723fb237f26ecff86e1b2d4e2e not found: ID does not exist" Jul 12 08:40:21.453346 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.453332 2558 scope.go:117] "RemoveContainer" containerID="45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f" Jul 12 08:40:21.453548 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:40:21.453533 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f\": container with ID starting with 45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f not found: ID does not exist" containerID="45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f" Jul 12 08:40:21.453586 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.453554 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f"} err="failed to get container status \"45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f\": rpc error: code = NotFound desc = could not find container \"45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f\": container with ID starting with 45d3d9e1915d40d83cc19bb880e0f6e3f4efb575057f35b2fb4b5ba70675752f not found: ID does not exist" Jul 12 08:40:21.459000 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.458949 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd"] Jul 12 08:40:21.462405 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:21.462386 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-34847c-openshift-default-66694dfbdf-nhkjd"] Jul 12 08:40:22.387021 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:22.386986 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" path="/var/lib/kubelet/pods/83b19ed4-20ee-4c61-bea4-ddd0de358a98/volumes" Jul 12 08:40:24.812900 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:24.812858 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:25.246803 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:25.246700 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:25.545769 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:25.545666 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:25.781659 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:25.781624 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:31.095093 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:31.095056 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:31.181204 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:31.181158 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:31.245675 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:31.245634 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:31.346000 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:31.345914 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:31.745504 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:31.745406 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:31.847721 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:31.847680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:35.467952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:35.467913 2558 patch_prober.go:28] interesting pod/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 08:40:35.468360 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:35.468001 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 08:40:35.481134 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:35.481104 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:37.506951 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:37.506917 2558 generic.go:356] "Generic (PLEG): container finished" podID="0af6fa82-e51f-49b4-8815-f75447058db2" containerID="c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c" exitCode=0 Jul 12 08:40:37.507377 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:37.506988 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" event={"ID":"0af6fa82-e51f-49b4-8815-f75447058db2","Type":"ContainerDied","Data":"c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c"} Jul 12 08:40:37.507377 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:37.507348 2558 scope.go:117] "RemoveContainer" containerID="c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c" Jul 12 08:40:37.508393 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:37.508359 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:37.508463 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:37.508433 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:37.508498 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:37.508466 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:38.513227 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:38.513193 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" event={"ID":"0af6fa82-e51f-49b4-8815-f75447058db2","Type":"ContainerStarted","Data":"69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564"} Jul 12 08:40:39.254562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:39.254520 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:39.254828 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:39.254808 2558 patch_prober.go:28] interesting pod/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 12 08:40:39.254880 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:39.254853 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 12 08:40:40.254411 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:40.254372 2558 patch_prober.go:28] interesting pod/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 12 08:40:40.254904 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:40.254427 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 12 08:40:41.254633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:41.254592 2558 patch_prober.go:28] interesting pod/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 12 08:40:41.255049 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:41.254660 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 12 08:40:42.254754 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:42.254715 2558 patch_prober.go:28] interesting pod/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 12 08:40:42.255142 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:42.254795 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 12 08:40:42.627199 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:42.627166 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:42.724927 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:42.724893 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:43.258166 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:43.258132 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:43.259400 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:43.259379 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:43.260573 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:43.260549 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:43.479403 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:43.479372 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:43.877795 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:43.877760 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:43.971511 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:43.971474 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:49.810112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:49.810075 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:49.869753 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:49.869720 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:50.057820 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:50.057790 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627"] Jul 12 08:40:50.057990 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:50.057969 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" containerID="cri-o://69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564" gracePeriod=30 Jul 12 08:40:50.086544 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:50.086454 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:50.432147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:50.432064 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:50.810878 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:50.810848 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:50.916138 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:50.916100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:51.011963 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:51.011932 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:51.104326 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:51.104252 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:55.307727 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.307698 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:55.447832 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447733 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-socket\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.447832 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447806 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af6fa82-e51f-49b4-8815-f75447058db2-istiod-ca-cert\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447846 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-data\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447862 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-certs\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447926 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af6fa82-e51f-49b4-8815-f75447058db2-istio-podinfo\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447958 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-envoy\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447987 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjkkk\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-kube-api-access-vjkkk\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448018 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.447985 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:55.448317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448033 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-istio-token\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448087 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-credential-socket\") pod \"0af6fa82-e51f-49b4-8815-f75447058db2\" (UID: \"0af6fa82-e51f-49b4-8815-f75447058db2\") " Jul 12 08:40:55.448317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448250 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0af6fa82-e51f-49b4-8815-f75447058db2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:40:55.448317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448268 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:55.448485 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448355 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.448485 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448446 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:55.448485 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448462 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af6fa82-e51f-49b4-8815-f75447058db2-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.448485 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448480 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.448887 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.448776 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-data" (OuterVolumeSpecName: "istio-data") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:55.450268 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.450243 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0af6fa82-e51f-49b4-8815-f75447058db2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:40:55.450268 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.450259 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:40:55.450538 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.450518 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-istio-token" (OuterVolumeSpecName: "istio-token") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:40:55.450636 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.450544 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-kube-api-access-vjkkk" (OuterVolumeSpecName: "kube-api-access-vjkkk") pod "0af6fa82-e51f-49b4-8815-f75447058db2" (UID: "0af6fa82-e51f-49b4-8815-f75447058db2"). InnerVolumeSpecName "kube-api-access-vjkkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:40:55.549562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.549538 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.549562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.549557 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af6fa82-e51f-49b4-8815-f75447058db2-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.549562 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.549567 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.549771 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.549575 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjkkk\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-kube-api-access-vjkkk\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.549771 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.549584 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af6fa82-e51f-49b4-8815-f75447058db2-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.549771 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.549592 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af6fa82-e51f-49b4-8815-f75447058db2-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:40:55.591685 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.591658 2558 generic.go:356] "Generic (PLEG): container finished" podID="0af6fa82-e51f-49b4-8815-f75447058db2" containerID="69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564" exitCode=0 Jul 12 08:40:55.591816 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.591705 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" Jul 12 08:40:55.591816 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.591738 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" event={"ID":"0af6fa82-e51f-49b4-8815-f75447058db2","Type":"ContainerDied","Data":"69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564"} Jul 12 08:40:55.591816 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.591814 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627" event={"ID":"0af6fa82-e51f-49b4-8815-f75447058db2","Type":"ContainerDied","Data":"d2a1c23f98e225e8c3aec50fa20a37816d0c4369af52fd4e4b2edb0eaca85291"} Jul 12 08:40:55.591955 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.591837 2558 scope.go:117] "RemoveContainer" containerID="69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564" Jul 12 08:40:55.600660 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.600642 2558 scope.go:117] "RemoveContainer" containerID="c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c" Jul 12 08:40:55.608098 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.608082 2558 scope.go:117] "RemoveContainer" containerID="69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564" Jul 12 08:40:55.608350 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:40:55.608329 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564\": container with ID starting with 69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564 not found: ID does not exist" containerID="69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564" Jul 12 08:40:55.608397 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.608357 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564"} err="failed to get container status \"69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564\": rpc error: code = NotFound desc = could not find container \"69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564\": container with ID starting with 69c899e58560714d3b3631a6d79774835fce6828c8dc92289cdf13fdf8ac0564 not found: ID does not exist" Jul 12 08:40:55.608397 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.608374 2558 scope.go:117] "RemoveContainer" containerID="c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c" Jul 12 08:40:55.608576 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:40:55.608556 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c\": container with ID starting with c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c not found: ID does not exist" containerID="c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c" Jul 12 08:40:55.608634 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.608588 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c"} err="failed to get container status \"c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c\": rpc error: code = NotFound desc = could not find container \"c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c\": container with ID starting with c198cf493187535717b83239f3bfc853a255113aad460f45866c2d7083d91e6c not found: ID does not exist" Jul 12 08:40:55.611918 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.611898 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627"] Jul 12 08:40:55.616526 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:55.616507 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-449923-openshift-default-7d96dc75b9-b2627"] Jul 12 08:40:56.387252 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:56.387210 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" path="/var/lib/kubelet/pods/0af6fa82-e51f-49b4-8815-f75447058db2/volumes" Jul 12 08:40:56.595298 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:56.595265 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:56.695137 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:56.695100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:56.848075 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:56.847990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:57.371700 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.371662 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:57.637651 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637576 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg"] Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637883 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637895 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637906 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637912 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637926 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637932 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637952 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" Jul 12 08:40:57.637996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.637958 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" Jul 12 08:40:57.638256 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.638003 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" Jul 12 08:40:57.638256 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.638012 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" Jul 12 08:40:57.638256 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.638018 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9ba3421-c2e1-4c27-b4df-7c615e7b8f96" containerName="istio-proxy" Jul 12 08:40:57.638256 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.638023 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af6fa82-e51f-49b4-8815-f75447058db2" containerName="istio-proxy" Jul 12 08:40:57.638256 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.638027 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" Jul 12 08:40:57.642481 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.642462 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.644875 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.644853 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-23d4ae-openshift-default-dockercfg-5ftz2\"" Jul 12 08:40:57.654279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.654253 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg"] Jul 12 08:40:57.666523 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666503 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-envoy\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666531 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-podinfo\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-credential-socket\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666572 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istiod-ca-cert\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666761 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-certs\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666761 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666669 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-socket\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666761 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666687 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx88h\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-kube-api-access-sx88h\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666761 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666738 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-data\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.666890 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.666808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-token\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.677735 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.677708 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:57.767473 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767445 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-envoy\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767480 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-podinfo\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767507 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-credential-socket\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767534 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istiod-ca-cert\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767566 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-certs\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767612 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-socket\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767911 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767655 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx88h\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-kube-api-access-sx88h\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767911 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767722 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-data\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.767911 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767779 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-token\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.768057 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.767945 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-socket\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.768217 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.768142 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-credential-socket\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.768399 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.768372 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-data\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.768524 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.768491 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-certs\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.768617 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.768592 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istiod-ca-cert\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.769927 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.769900 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:40:57.770147 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.770124 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-envoy\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.770391 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.770375 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-podinfo\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.779992 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.779970 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-token\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.780242 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.780222 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx88h\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-kube-api-access-sx88h\") pod \"e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:57.954633 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:57.954540 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:58.086235 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.086210 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg"] Jul 12 08:40:58.087134 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:40:58.087102 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd63c03c_8cfa_4f8f_8804_f35c9ba7b280.slice/crio-98ae1336f1886271d33432fb3ae77eeff26d2b9c92e6150842a40f7fe5af0487 WatchSource:0}: Error finding container 98ae1336f1886271d33432fb3ae77eeff26d2b9c92e6150842a40f7fe5af0487: Status 404 returned error can't find the container with id 98ae1336f1886271d33432fb3ae77eeff26d2b9c92e6150842a40f7fe5af0487 Jul 12 08:40:58.089235 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.089204 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:58.089317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.089268 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:58.089317 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.089298 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:40:58.605660 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.605621 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" event={"ID":"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280","Type":"ContainerStarted","Data":"5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a"} Jul 12 08:40:58.605888 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.605683 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" event={"ID":"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280","Type":"ContainerStarted","Data":"98ae1336f1886271d33432fb3ae77eeff26d2b9c92e6150842a40f7fe5af0487"} Jul 12 08:40:58.628006 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.627949 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podStartSLOduration=1.6279293350000001 podStartE2EDuration="1.627929335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:40:58.626326505 +0000 UTC m=+2043.071811648" watchObservedRunningTime="2026-07-12 08:40:58.627929335 +0000 UTC m=+2043.073414469" Jul 12 08:40:58.955398 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:58.955310 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:40:59.959602 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:40:59.959573 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:00.486448 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:00.486403 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:00.565610 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:00.565576 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:00.614985 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:00.614880 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:00.619775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:00.619727 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:00.631656 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:00.631635 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:00.732017 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:00.731986 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:01.164521 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:01.164492 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:01.260077 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:01.260046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:04.935494 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:04.935464 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:05.252118 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.252041 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:05.523084 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.523014 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt"] Jul 12 08:41:05.523322 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.523310 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" Jul 12 08:41:05.523367 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.523324 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b19ed4-20ee-4c61-bea4-ddd0de358a98" containerName="istio-proxy" Jul 12 08:41:05.526244 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.526227 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.528444 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.528424 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b4359a-openshift-default-dockercfg-wpszw\"" Jul 12 08:41:05.537433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.537411 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt"] Jul 12 08:41:05.561600 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.561572 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:05.631396 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631354 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-envoy\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631583 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-credential-socket\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631583 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631509 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-istio-token\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631583 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hbcq\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-kube-api-access-4hbcq\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631628 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0ac39f1-46de-45b1-add9-72908acfce16-istiod-ca-cert\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631681 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-certs\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631704 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-data\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631734 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0ac39f1-46de-45b1-add9-72908acfce16-istio-podinfo\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.631996 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.631777 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-socket\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.732902 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.732873 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-credential-socket\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733071 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.732910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-istio-token\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733071 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.732926 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hbcq\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-kube-api-access-4hbcq\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733071 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733050 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0ac39f1-46de-45b1-add9-72908acfce16-istiod-ca-cert\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-certs\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733134 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-data\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0ac39f1-46de-45b1-add9-72908acfce16-istio-podinfo\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733193 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-socket\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733264 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733255 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-envoy\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733541 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733340 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-credential-socket\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733541 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733447 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-certs\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733541 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733484 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-data\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733704 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733661 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-socket\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.733897 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.733875 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0ac39f1-46de-45b1-add9-72908acfce16-istiod-ca-cert\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.735536 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.735509 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-envoy\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.735735 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.735716 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0ac39f1-46de-45b1-add9-72908acfce16-istio-podinfo\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.740254 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.740231 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-istio-token\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.740661 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.740641 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hbcq\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-kube-api-access-4hbcq\") pod \"e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.837663 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.837636 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:05.970844 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.970816 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt"] Jul 12 08:41:05.972989 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:41:05.972956 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ac39f1_46de_45b1_add9_72908acfce16.slice/crio-5845b090ac76fee73de2db968dc1800dbd3a303a5b31c6735f3a969d38ec8384 WatchSource:0}: Error finding container 5845b090ac76fee73de2db968dc1800dbd3a303a5b31c6735f3a969d38ec8384: Status 404 returned error can't find the container with id 5845b090ac76fee73de2db968dc1800dbd3a303a5b31c6735f3a969d38ec8384 Jul 12 08:41:05.975519 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.975478 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:05.975615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.975540 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:05.975615 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:05.975570 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:06.080884 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:06.080844 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:06.641679 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:06.641642 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" event={"ID":"b0ac39f1-46de-45b1-add9-72908acfce16","Type":"ContainerStarted","Data":"5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0"} Jul 12 08:41:06.641881 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:06.641693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" event={"ID":"b0ac39f1-46de-45b1-add9-72908acfce16","Type":"ContainerStarted","Data":"5845b090ac76fee73de2db968dc1800dbd3a303a5b31c6735f3a969d38ec8384"} Jul 12 08:41:06.665357 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:06.665308 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" podStartSLOduration=1.6652926670000001 podStartE2EDuration="1.665292667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:41:06.665017171 +0000 UTC m=+2051.110502302" watchObservedRunningTime="2026-07-12 08:41:06.665292667 +0000 UTC m=+2051.110777797" Jul 12 08:41:06.838362 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:06.838329 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:07.844102 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:07.844074 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:08.541183 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:08.541144 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:08.636219 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:08.636181 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:08.653253 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:08.653211 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:08.654173 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:08.653954 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:08.703212 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:08.703175 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:08.943151 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:08.943122 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:09.186112 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:09.186082 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:09.286769 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:09.286662 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:13.007193 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:13.007158 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:15.616807 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.616775 2558 patch_prober.go:28] interesting pod/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 08:41:15.617218 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.616829 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 08:41:15.683201 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.683169 2558 generic.go:356] "Generic (PLEG): container finished" podID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerID="5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a" exitCode=0 Jul 12 08:41:15.683398 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.683233 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" event={"ID":"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280","Type":"ContainerDied","Data":"5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a"} Jul 12 08:41:15.683672 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.683654 2558 scope.go:117] "RemoveContainer" containerID="5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a" Jul 12 08:41:15.684710 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.684680 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:15.684829 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.684760 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:15.684829 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:15.684794 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:16.689287 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:16.689251 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" event={"ID":"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280","Type":"ContainerStarted","Data":"093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4"} Jul 12 08:41:16.955056 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:16.954871 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:16.955247 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:16.955163 2558 patch_prober.go:28] interesting pod/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 08:41:16.955247 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:16.955222 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 08:41:17.955332 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:17.955244 2558 patch_prober.go:28] interesting pod/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 08:41:17.955939 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:17.955906 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 08:41:18.955049 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:18.955005 2558 patch_prober.go:28] interesting pod/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 08:41:18.955297 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:18.955072 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 08:41:19.955290 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:19.955250 2558 patch_prober.go:28] interesting pod/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 08:41:19.955671 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:19.955308 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 08:41:20.958999 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:20.958968 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:20.959467 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:20.959361 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:20.959945 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:20.959925 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:41:21.865343 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:21.865305 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:21.967629 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:21.967598 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:22.721477 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:22.721444 2558 generic.go:356] "Generic (PLEG): container finished" podID="b0ac39f1-46de-45b1-add9-72908acfce16" containerID="5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0" exitCode=0 Jul 12 08:41:22.721696 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:22.721523 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" event={"ID":"b0ac39f1-46de-45b1-add9-72908acfce16","Type":"ContainerDied","Data":"5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0"} Jul 12 08:41:22.722030 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:22.722010 2558 scope.go:117] "RemoveContainer" containerID="5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0" Jul 12 08:41:22.723000 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:22.722975 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:22.723072 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:22.723037 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:22.723072 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:22.723067 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:41:23.653100 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:23.653060 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:23.731784 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:23.731437 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" event={"ID":"b0ac39f1-46de-45b1-add9-72908acfce16","Type":"ContainerStarted","Data":"55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b"} Jul 12 08:41:23.839330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:23.838922 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:23.839330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:23.839036 2558 patch_prober.go:28] interesting pod/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 12 08:41:23.839330 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:23.839080 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 12 08:41:24.838648 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:24.838615 2558 patch_prober.go:28] interesting pod/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 12 08:41:24.839644 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:24.839228 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 12 08:41:25.838723 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:25.838681 2558 patch_prober.go:28] interesting pod/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 12 08:41:25.839234 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:25.838776 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 12 08:41:26.838107 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:26.838068 2558 patch_prober.go:28] interesting pod/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 12 08:41:26.838320 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:26.838135 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 12 08:41:27.396967 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:27.396933 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:27.497944 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:27.497914 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:27.842038 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:27.842013 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:27.842361 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:27.842338 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:27.842914 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:27.842896 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:41:29.999055 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:29.999019 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:30.104419 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:30.104389 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:43.293173 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:43.293135 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:56.226879 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:56.226842 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:41:56.231551 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:41:56.231529 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chzp5_47fe965a-1957-408e-8cf6-725c09aeccbc/ovn-acl-logging/0.log" Jul 12 08:44:17.818331 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:17.818293 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:18.196406 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:18.196284 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:18.296929 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:18.296895 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:18.467087 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:18.467001 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:24.027790 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:24.027731 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:24.128308 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:24.128277 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:24.256889 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:24.256850 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg"] Jul 12 08:44:24.257170 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:24.257139 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" containerID="cri-o://093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4" gracePeriod=30 Jul 12 08:44:24.276798 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:24.276725 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:24.643628 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:24.643594 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:25.021814 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:25.021701 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:25.122391 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:25.122355 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:25.223004 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:25.222972 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:25.321768 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:25.321718 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:29.507143 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.507111 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:44:29.555379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.555286 2558 generic.go:356] "Generic (PLEG): container finished" podID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerID="093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4" exitCode=0 Jul 12 08:44:29.555379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.555338 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" Jul 12 08:44:29.555379 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.555349 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" event={"ID":"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280","Type":"ContainerDied","Data":"093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4"} Jul 12 08:44:29.555666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.555446 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg" event={"ID":"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280","Type":"ContainerDied","Data":"98ae1336f1886271d33432fb3ae77eeff26d2b9c92e6150842a40f7fe5af0487"} Jul 12 08:44:29.555666 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.555477 2558 scope.go:117] "RemoveContainer" containerID="093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4" Jul 12 08:44:29.563789 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.563765 2558 scope.go:117] "RemoveContainer" containerID="5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a" Jul 12 08:44:29.571249 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.571233 2558 scope.go:117] "RemoveContainer" containerID="093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4" Jul 12 08:44:29.571495 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:44:29.571475 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4\": container with ID starting with 093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4 not found: ID does not exist" containerID="093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4" Jul 12 08:44:29.571540 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.571504 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4"} err="failed to get container status \"093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4\": rpc error: code = NotFound desc = could not find container \"093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4\": container with ID starting with 093aa211125269b26604a4ae43a714fe9ee2e688d2b45f5586a24b89cf7a6ff4 not found: ID does not exist" Jul 12 08:44:29.571540 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.571522 2558 scope.go:117] "RemoveContainer" containerID="5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a" Jul 12 08:44:29.571760 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:44:29.571725 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a\": container with ID starting with 5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a not found: ID does not exist" containerID="5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a" Jul 12 08:44:29.571805 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.571764 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a"} err="failed to get container status \"5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a\": rpc error: code = NotFound desc = could not find container \"5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a\": container with ID starting with 5022dbe882b8883335ae77c1e6f30736949d24577d0479e1471df68377b4f14a not found: ID does not exist" Jul 12 08:44:29.604766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604717 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-podinfo\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.604916 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604789 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-credential-socket\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.604916 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604812 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-data\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.604916 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604839 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-certs\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.604916 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604875 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istiod-ca-cert\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.604916 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604911 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-token\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.605168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.604951 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-socket\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.605168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605023 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-envoy\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.605168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605057 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx88h\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-kube-api-access-sx88h\") pod \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\" (UID: \"dd63c03c-8cfa-4f8f-8804-f35c9ba7b280\") " Jul 12 08:44:29.605168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605072 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:29.605370 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605186 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:29.605370 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605333 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.605370 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605350 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.605528 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605386 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:29.605528 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605312 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:44:29.606030 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.605995 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:29.607048 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.607025 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:44:29.607205 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.607185 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:44:29.607275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.607255 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-kube-api-access-sx88h" (OuterVolumeSpecName: "kube-api-access-sx88h") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "kube-api-access-sx88h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:44:29.607275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.607267 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" (UID: "dd63c03c-8cfa-4f8f-8804-f35c9ba7b280"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:29.705709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705677 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx88h\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-kube-api-access-sx88h\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.705709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705706 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.705709 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705716 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.705965 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705724 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.705965 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705732 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.705965 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705761 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.705965 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.705773 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:29.876384 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.876352 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg"] Jul 12 08:44:29.881371 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:29.881346 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-23d4ae-openshift-default-6b9d8c7fcf-7l9gg"] Jul 12 08:44:30.387909 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:30.387878 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" path="/var/lib/kubelet/pods/dd63c03c-8cfa-4f8f-8804-f35c9ba7b280/volumes" Jul 12 08:44:30.668701 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:30.668608 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:30.735947 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:30.735905 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:30.975959 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:30.975855 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt"] Jul 12 08:44:30.976157 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:30.976110 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" containerID="cri-o://55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b" gracePeriod=30 Jul 12 08:44:30.989400 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:30.989373 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:31.699874 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:31.699840 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b4867c6ff-hzlwv_c9093c26-8655-475a-8fba-01d6b174a34a/authorino/0.log" Jul 12 08:44:36.216614 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.216589 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:44:36.260202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260123 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hbcq\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-kube-api-access-4hbcq\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260202 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260189 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-certs\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260223 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-envoy\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260240 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-istio-token\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260277 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-credential-socket\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260301 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0ac39f1-46de-45b1-add9-72908acfce16-istio-podinfo\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260325 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-data\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260355 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0ac39f1-46de-45b1-add9-72908acfce16-istiod-ca-cert\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260421 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260387 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-socket\") pod \"b0ac39f1-46de-45b1-add9-72908acfce16\" (UID: \"b0ac39f1-46de-45b1-add9-72908acfce16\") " Jul 12 08:44:36.260826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260532 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:36.260826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260702 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-certs\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.260826 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260731 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:36.260995 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.260841 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:36.261050 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.261006 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0ac39f1-46de-45b1-add9-72908acfce16-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:44:36.261585 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.261558 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:36.262462 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.262438 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:44:36.262598 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.262517 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-kube-api-access-4hbcq" (OuterVolumeSpecName: "kube-api-access-4hbcq") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "kube-api-access-4hbcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:44:36.262963 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.262935 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:44:36.263054 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.262955 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0ac39f1-46de-45b1-add9-72908acfce16-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0ac39f1-46de-45b1-add9-72908acfce16" (UID: "b0ac39f1-46de-45b1-add9-72908acfce16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:44:36.361099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361067 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-envoy\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361092 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-istio-token\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361099 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361101 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-credential-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361334 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361111 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0ac39f1-46de-45b1-add9-72908acfce16-istio-podinfo\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361334 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361120 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-istio-data\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361334 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361127 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0ac39f1-46de-45b1-add9-72908acfce16-istiod-ca-cert\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361334 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361135 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0ac39f1-46de-45b1-add9-72908acfce16-workload-socket\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.361334 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.361144 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hbcq\" (UniqueName: \"kubernetes.io/projected/b0ac39f1-46de-45b1-add9-72908acfce16-kube-api-access-4hbcq\") on node \"ip-10-0-139-48.ec2.internal\" DevicePath \"\"" Jul 12 08:44:36.447155 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.447124 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-g5w7p_ed416917-3fb6-43a5-992e-2fb8367ad43a/server/0.log" Jul 12 08:44:36.550926 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.550850 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fgbrl_f398045d-0d89-4774-bd23-890d1892bd7f/manager/0.log" Jul 12 08:44:36.586142 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.586111 2558 generic.go:356] "Generic (PLEG): container finished" podID="b0ac39f1-46de-45b1-add9-72908acfce16" containerID="55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b" exitCode=0 Jul 12 08:44:36.586288 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.586153 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" Jul 12 08:44:36.586288 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.586193 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" event={"ID":"b0ac39f1-46de-45b1-add9-72908acfce16","Type":"ContainerDied","Data":"55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b"} Jul 12 08:44:36.586288 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.586249 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt" event={"ID":"b0ac39f1-46de-45b1-add9-72908acfce16","Type":"ContainerDied","Data":"5845b090ac76fee73de2db968dc1800dbd3a303a5b31c6735f3a969d38ec8384"} Jul 12 08:44:36.586288 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.586273 2558 scope.go:117] "RemoveContainer" containerID="55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b" Jul 12 08:44:36.594491 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.594473 2558 scope.go:117] "RemoveContainer" containerID="5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0" Jul 12 08:44:36.601876 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.601854 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt"] Jul 12 08:44:36.602067 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.602044 2558 scope.go:117] "RemoveContainer" containerID="55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b" Jul 12 08:44:36.602321 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:44:36.602302 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b\": container with ID starting with 55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b not found: ID does not exist" containerID="55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b" Jul 12 08:44:36.602399 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.602334 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b"} err="failed to get container status \"55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b\": rpc error: code = NotFound desc = could not find container \"55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b\": container with ID starting with 55d8afb0c2b1fa04844d729f56a9dc405d1233227c5a0c08bdbfa1132e56dd0b not found: ID does not exist" Jul 12 08:44:36.602399 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.602357 2558 scope.go:117] "RemoveContainer" containerID="5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0" Jul 12 08:44:36.602617 ip-10-0-139-48 kubenswrapper[2558]: E0712 08:44:36.602600 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0\": container with ID starting with 5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0 not found: ID does not exist" containerID="5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0" Jul 12 08:44:36.602661 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.602623 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0"} err="failed to get container status \"5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0\": rpc error: code = NotFound desc = could not find container \"5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0\": container with ID starting with 5c98f286ab96816675fcc0d3feb60ff0f9e5fe0fbd818426ccd83125e762f9e0 not found: ID does not exist" Jul 12 08:44:36.606136 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.606116 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b4359a-openshift-default-7bd588df7f-4fwnt"] Jul 12 08:44:36.805891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.805816 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-595c444865-rwxr8_a2373797-ca9d-491c-ae97-cc4fdaa0f51a/copy-manifests/0.log" Jul 12 08:44:36.812473 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:36.812449 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-595c444865-rwxr8_a2373797-ca9d-491c-ae97-cc4fdaa0f51a/manager/0.log" Jul 12 08:44:38.178433 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:38.178402 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b4867c6ff-hzlwv_c9093c26-8655-475a-8fba-01d6b174a34a/authorino/0.log" Jul 12 08:44:38.387687 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:38.387653 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" path="/var/lib/kubelet/pods/b0ac39f1-46de-45b1-add9-72908acfce16/volumes" Jul 12 08:44:39.307591 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:39.307564 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v2qnt_0b8c43c1-5263-4086-baf0-c7dcfb1593c3/istio-proxy/0.log" Jul 12 08:44:39.631997 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:39.631973 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cfff4589-kcbv7_0c221d97-63d9-4eae-b30f-bb2b5bf91714/kube-auth-proxy/0.log" Jul 12 08:44:39.744373 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:39.744343 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/1.log" Jul 12 08:44:40.726940 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:40.726913 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvfw92_9c16538c-64fe-4675-a8a6-1b6352ca5323/main/0.log" Jul 12 08:44:40.943924 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:40.943894 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-l6mfj_2bc326c5-7428-4f51-af73-2a6058c07bb9/main/0.log" Jul 12 08:44:45.308891 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.308859 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ndtc/must-gather-9qvh2"] Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309140 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309151 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309167 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309173 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309231 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309242 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" Jul 12 08:44:45.309275 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309249 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" Jul 12 08:44:45.309495 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309288 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" Jul 12 08:44:45.309495 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309294 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0ac39f1-46de-45b1-add9-72908acfce16" containerName="istio-proxy" Jul 12 08:44:45.309495 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309311 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" Jul 12 08:44:45.309495 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309316 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" Jul 12 08:44:45.309495 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.309362 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd63c03c-8cfa-4f8f-8804-f35c9ba7b280" containerName="istio-proxy" Jul 12 08:44:45.313853 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.313833 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.316215 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.316188 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ndtc\"/\"default-dockercfg-9k25p\"" Jul 12 08:44:45.316215 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.316204 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ndtc\"/\"kube-root-ca.crt\"" Jul 12 08:44:45.316430 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.316189 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ndtc\"/\"openshift-service-ca.crt\"" Jul 12 08:44:45.321776 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.321754 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ndtc/must-gather-9qvh2"] Jul 12 08:44:45.328123 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.328101 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:45.334642 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.334618 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2hzt\" (UniqueName: \"kubernetes.io/projected/b1b5922f-bb8f-48c5-935e-6ad6f37eb038-kube-api-access-b2hzt\") pod \"must-gather-9qvh2\" (UID: \"b1b5922f-bb8f-48c5-935e-6ad6f37eb038\") " pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.334735 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.334660 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b1b5922f-bb8f-48c5-935e-6ad6f37eb038-must-gather-output\") pod \"must-gather-9qvh2\" (UID: \"b1b5922f-bb8f-48c5-935e-6ad6f37eb038\") " pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.398628 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.398569 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:45.435700 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.435672 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2hzt\" (UniqueName: \"kubernetes.io/projected/b1b5922f-bb8f-48c5-935e-6ad6f37eb038-kube-api-access-b2hzt\") pod \"must-gather-9qvh2\" (UID: \"b1b5922f-bb8f-48c5-935e-6ad6f37eb038\") " pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.435847 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.435724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b1b5922f-bb8f-48c5-935e-6ad6f37eb038-must-gather-output\") pod \"must-gather-9qvh2\" (UID: \"b1b5922f-bb8f-48c5-935e-6ad6f37eb038\") " pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.436022 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.436002 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b1b5922f-bb8f-48c5-935e-6ad6f37eb038-must-gather-output\") pod \"must-gather-9qvh2\" (UID: \"b1b5922f-bb8f-48c5-935e-6ad6f37eb038\") " pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.446868 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.446848 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2hzt\" (UniqueName: \"kubernetes.io/projected/b1b5922f-bb8f-48c5-935e-6ad6f37eb038-kube-api-access-b2hzt\") pod \"must-gather-9qvh2\" (UID: \"b1b5922f-bb8f-48c5-935e-6ad6f37eb038\") " pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.623341 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.623253 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ndtc/must-gather-9qvh2" Jul 12 08:44:45.748425 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.748391 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ndtc/must-gather-9qvh2"] Jul 12 08:44:45.751653 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:44:45.751623 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1b5922f_bb8f_48c5_935e_6ad6f37eb038.slice/crio-0e4e37f75f06a2f6e5fe5caeee7100295d3c3eff7d5443935f5dcdd0575a66c0 WatchSource:0}: Error finding container 0e4e37f75f06a2f6e5fe5caeee7100295d3c3eff7d5443935f5dcdd0575a66c0: Status 404 returned error can't find the container with id 0e4e37f75f06a2f6e5fe5caeee7100295d3c3eff7d5443935f5dcdd0575a66c0 Jul 12 08:44:45.753910 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:45.753887 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:44:46.629925 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:46.629890 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndtc/must-gather-9qvh2" event={"ID":"b1b5922f-bb8f-48c5-935e-6ad6f37eb038","Type":"ContainerStarted","Data":"0e4e37f75f06a2f6e5fe5caeee7100295d3c3eff7d5443935f5dcdd0575a66c0"} Jul 12 08:44:47.637961 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:47.637911 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndtc/must-gather-9qvh2" event={"ID":"b1b5922f-bb8f-48c5-935e-6ad6f37eb038","Type":"ContainerStarted","Data":"113fba06b52fe3ca7adfc9eea2874265aebc1fcb91a844b62b9da7d348845669"} Jul 12 08:44:47.638424 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:47.637975 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndtc/must-gather-9qvh2" event={"ID":"b1b5922f-bb8f-48c5-935e-6ad6f37eb038","Type":"ContainerStarted","Data":"8d6cfd3f3eb27585e33065f34900eba8feb1cea695f778e037118e56868b6359"} Jul 12 08:44:48.101280 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:48.101251 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8gf2t_083f72d9-be4f-4fcb-a2a3-19fd4c746884/global-pull-secret-syncer/0.log" Jul 12 08:44:48.227517 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:48.227483 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" Jul 12 08:44:48.355434 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:48.355362 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-48.ec2.internal_477ee7992b12ed0719fbc7684ecfecc6/haproxy/0.log" Jul 12 08:44:53.227766 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:53.227186 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b4867c6ff-hzlwv_c9093c26-8655-475a-8fba-01d6b174a34a/authorino/0.log" Jul 12 08:44:55.924670 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:55.924630 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kgbnt_9af93d0b-cb4d-4086-bb54-731f1c277eff/node-exporter/0.log" Jul 12 08:44:55.947062 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:55.947022 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kgbnt_9af93d0b-cb4d-4086-bb54-731f1c277eff/kube-rbac-proxy/0.log" Jul 12 08:44:55.965897 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:55.965793 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kgbnt_9af93d0b-cb4d-4086-bb54-731f1c277eff/init-textfile/0.log" Jul 12 08:44:57.332650 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:57.332610 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4gg28_06f5868c-0a31-4d2f-badf-3c0fa2a950a0/konnectivity-agent/0.log" limit=1024 Jul 12 08:44:58.057992 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.057920 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ndtc/must-gather-9qvh2" podStartSLOduration=12.226194296 podStartE2EDuration="13.057898065s" totalImagesPullingTime="831.703769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:44:47.654099791 +0000 UTC m=+2272.099584926" watchObservedRunningTime="2026-07-12 08:44:58.057898065 +0000 UTC m=+2282.503383196" Jul 12 08:44:58.060033 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.060008 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl"] Jul 12 08:44:58.068035 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.068008 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.073300 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.073272 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl"] Jul 12 08:44:58.166402 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.166352 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-podres\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.166585 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.166419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-lib-modules\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.166585 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.166458 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc2c4\" (UniqueName: \"kubernetes.io/projected/f7aeb03a-8982-4297-9d52-965ed5a206cd-kube-api-access-cc2c4\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.166585 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.166495 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-sys\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.166585 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.166523 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-proc\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.267630 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.267592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-podres\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.267852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.267661 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-lib-modules\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.267852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.267703 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc2c4\" (UniqueName: \"kubernetes.io/projected/f7aeb03a-8982-4297-9d52-965ed5a206cd-kube-api-access-cc2c4\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.267852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.267768 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-sys\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.267852 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.267797 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-proc\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.268105 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.267963 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-proc\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.268105 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.268082 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-podres\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.268226 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.268166 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-lib-modules\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.268419 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.268394 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7aeb03a-8982-4297-9d52-965ed5a206cd-sys\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.276514 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.276488 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc2c4\" (UniqueName: \"kubernetes.io/projected/f7aeb03a-8982-4297-9d52-965ed5a206cd-kube-api-access-cc2c4\") pod \"perf-node-gather-daemonset-fsmvl\" (UID: \"f7aeb03a-8982-4297-9d52-965ed5a206cd\") " pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.383042 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.382956 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:58.569693 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.569658 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl"] Jul 12 08:44:58.573049 ip-10-0-139-48 kubenswrapper[2558]: W0712 08:44:58.573016 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7aeb03a_8982_4297_9d52_965ed5a206cd.slice/crio-f32acc977fa58dcf2dba38465d2c35171139d33c91610fed353ce234d3e44dfa WatchSource:0}: Error finding container f32acc977fa58dcf2dba38465d2c35171139d33c91610fed353ce234d3e44dfa: Status 404 returned error can't find the container with id f32acc977fa58dcf2dba38465d2c35171139d33c91610fed353ce234d3e44dfa Jul 12 08:44:58.710775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:58.710639 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" event={"ID":"f7aeb03a-8982-4297-9d52-965ed5a206cd","Type":"ContainerStarted","Data":"f32acc977fa58dcf2dba38465d2c35171139d33c91610fed353ce234d3e44dfa"} Jul 12 08:44:59.719081 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:59.719022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" event={"ID":"f7aeb03a-8982-4297-9d52-965ed5a206cd","Type":"ContainerStarted","Data":"7c841894a78c67b6ef0343ab16751501a15b0ab08d3cf9002c7c12d7f4149684"} Jul 12 08:44:59.719546 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:59.719406 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:44:59.734389 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:44:59.734333 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" podStartSLOduration=1.73431617 podStartE2EDuration="1.73431617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:44:59.733482613 +0000 UTC m=+2284.178967744" watchObservedRunningTime="2026-07-12 08:44:59.73431617 +0000 UTC m=+2284.179801301" Jul 12 08:45:00.432265 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:00.432227 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-grhbr_8148b2db-3b6c-47ed-ba45-fcd50dd4d57d/dns/0.log" Jul 12 08:45:00.452997 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:00.452964 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-grhbr_8148b2db-3b6c-47ed-ba45-fcd50dd4d57d/kube-rbac-proxy/0.log" Jul 12 08:45:00.604279 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:00.604251 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zgdp2_3a49d418-33d8-4566-aed2-dad75f7401bb/dns-node-resolver/0.log" Jul 12 08:45:01.137952 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:01.137924 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lnw96_902ddd8c-1f3c-423f-bb33-498fcd687380/node-ca/0.log" Jul 12 08:45:02.054466 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:02.054433 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v2qnt_0b8c43c1-5263-4086-baf0-c7dcfb1593c3/istio-proxy/0.log" Jul 12 08:45:02.271985 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:02.271835 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cfff4589-kcbv7_0c221d97-63d9-4eae-b30f-bb2b5bf91714/kube-auth-proxy/0.log" Jul 12 08:45:02.301160 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:02.301126 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/0.log" Jul 12 08:45:02.340045 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:02.339977 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/1.log" Jul 12 08:45:03.126077 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:03.126045 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l8z9l_09bc5f43-1026-4cdc-8a41-eae824bd5d7c/serve-healthcheck-canary/0.log" Jul 12 08:45:03.795252 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:03.795216 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g25qv_f6138865-5794-481a-a099-18afed93496b/kube-rbac-proxy/0.log" Jul 12 08:45:03.814775 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:03.814751 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g25qv_f6138865-5794-481a-a099-18afed93496b/exporter/0.log" Jul 12 08:45:03.836862 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:03.836833 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g25qv_f6138865-5794-481a-a099-18afed93496b/extractor/0.log" Jul 12 08:45:05.733599 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:05.733562 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ndtc/perf-node-gather-daemonset-fsmvl" Jul 12 08:45:06.191527 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:06.191494 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-g5w7p_ed416917-3fb6-43a5-992e-2fb8367ad43a/server/0.log" Jul 12 08:45:06.269449 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:06.269405 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fgbrl_f398045d-0d89-4774-bd23-890d1892bd7f/manager/0.log" Jul 12 08:45:06.409770 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:06.409718 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-595c444865-rwxr8_a2373797-ca9d-491c-ae97-cc4fdaa0f51a/manager/0.log" Jul 12 08:45:06.428420 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:06.428392 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-595c444865-rwxr8_a2373797-ca9d-491c-ae97-cc4fdaa0f51a/copy-manifests/0.log" Jul 12 08:45:07.921352 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:07.921322 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59795878b4-8v8tg_752320ec-efe9-40c9-82c2-6d1f3a965a61/manager/0.log" Jul 12 08:45:08.406211 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:08.406182 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v2qnt_0b8c43c1-5263-4086-baf0-c7dcfb1593c3/istio-proxy/0.log" Jul 12 08:45:08.604042 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:08.604013 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cfff4589-kcbv7_0c221d97-63d9-4eae-b30f-bb2b5bf91714/kube-auth-proxy/0.log" Jul 12 08:45:08.621785 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:08.621754 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/0.log" Jul 12 08:45:08.658641 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:08.658606 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/1.log" Jul 12 08:45:09.927534 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:09.927506 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v2qnt_0b8c43c1-5263-4086-baf0-c7dcfb1593c3/istio-proxy/0.log" Jul 12 08:45:10.120769 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:10.120708 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cfff4589-kcbv7_0c221d97-63d9-4eae-b30f-bb2b5bf91714/kube-auth-proxy/0.log" Jul 12 08:45:10.140682 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:10.140651 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/0.log" Jul 12 08:45:10.179940 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:10.179849 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz94c_decfcf79-5029-46b2-8a3b-9d70fbfc5ee3/istio-proxy/1.log" Jul 12 08:45:12.880168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:12.880135 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-22zts_4e3059d4-f150-4292-9844-224920c3ac11/migrator/0.log" Jul 12 08:45:12.899471 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:12.899442 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-22zts_4e3059d4-f150-4292-9844-224920c3ac11/graceful-termination/0.log" Jul 12 08:45:14.242652 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.242616 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/kube-multus-additional-cni-plugins/0.log" Jul 12 08:45:14.263130 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.263099 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/egress-router-binary-copy/0.log" Jul 12 08:45:14.286770 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.286725 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/cni-plugins/0.log" Jul 12 08:45:14.307300 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.307272 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/bond-cni-plugin/0.log" Jul 12 08:45:14.328129 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.328097 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/routeoverride-cni/0.log" Jul 12 08:45:14.347611 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.347587 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/whereabouts-cni-bincopy/0.log" Jul 12 08:45:14.366168 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.366126 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29ggz_fa000e92-7ae2-4662-aa9f-11a3c2ff42cd/whereabouts-cni/0.log" Jul 12 08:45:14.719080 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.719048 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xb5hm_e7600187-1b9f-4f03-bf5b-6c6d91140376/kube-multus/0.log" Jul 12 08:45:14.826668 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.826604 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gjm7q_1fe07317-22f5-4710-9349-22d9672475e1/network-metrics-daemon/0.log" Jul 12 08:45:14.845987 ip-10-0-139-48 kubenswrapper[2558]: I0712 08:45:14.845961 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gjm7q_1fe07317-22f5-4710-9349-22d9672475e1/kube-rbac-proxy/0.log"