Jul 16 10:32:20.608202 ip-10-0-138-186 systemd[1]: Starting Kubernetes Kubelet... Jul 16 10:32:21.108509 ip-10-0-138-186 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 10:32:21.108509 ip-10-0-138-186 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 10:32:21.108509 ip-10-0-138-186 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 10:32:21.109087 ip-10-0-138-186 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 10:32:21.109087 ip-10-0-138-186 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 10:32:21.110216 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.110111 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 10:32:21.114256 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114241 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:21.114256 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114255 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114259 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114263 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114266 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114269 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114272 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114275 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114278 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114281 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114285 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114288 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114290 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114293 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114296 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114298 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114301 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114304 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114307 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114310 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114312 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:21.114320 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114315 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114318 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114321 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114324 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114328 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114332 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114336 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114339 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114341 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114344 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114347 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114349 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114352 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114355 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114357 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114360 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114362 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114365 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114368 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:21.114812 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114371 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114373 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114376 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114378 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114381 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114383 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114386 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114388 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114392 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114396 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114399 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114401 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114404 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114407 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114410 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114414 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114418 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114420 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114423 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114427 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:21.115271 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114429 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114432 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114435 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114438 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114440 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114443 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114446 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114448 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114451 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114459 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114462 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114465 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114467 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114470 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114474 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114476 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114479 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114481 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114485 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114489 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:21.115756 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114492 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114494 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114497 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114499 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114502 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114504 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114507 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114511 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114514 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114516 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114519 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114521 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114524 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114527 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114529 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114532 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114534 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114537 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.114539 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115242 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:21.116253 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115248 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115251 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115255 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115258 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115261 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115264 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115267 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115269 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115272 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115275 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115278 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115280 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115283 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115286 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115288 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115291 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115294 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115296 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115299 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115302 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:21.116718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115305 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115308 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115310 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115313 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115315 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115319 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115323 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115326 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115329 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115332 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115334 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115337 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115339 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115342 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115345 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115351 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115354 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115358 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115362 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:21.117252 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115366 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115369 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115372 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115374 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115377 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115380 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115383 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115386 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115388 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115392 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115394 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115397 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115400 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115403 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115405 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115408 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115410 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115413 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115415 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115418 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:21.117705 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115421 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115424 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115426 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115429 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115431 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115434 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115439 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115441 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115462 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115473 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115476 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115479 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115482 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115485 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115487 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115490 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115493 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115495 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115498 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:21.118208 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115501 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115504 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115506 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115510 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115512 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115515 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115518 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115520 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115523 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115526 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115528 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115531 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115533 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115536 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115539 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115541 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115544 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115546 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115549 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115553 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:21.118678 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115942 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115947 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115951 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115954 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115957 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115960 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115962 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115965 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115968 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115970 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115973 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115975 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115978 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115980 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115983 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115985 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115988 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115991 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115994 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:21.119167 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.115997 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116000 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116004 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116008 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116011 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116014 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116017 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116019 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116022 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116025 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116027 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116032 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116037 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116040 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116043 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116045 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116048 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116051 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116054 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:21.119621 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116056 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116059 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116061 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116064 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116066 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116069 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116073 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116076 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116079 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116082 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116085 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116089 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116092 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116094 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116097 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116101 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116103 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116106 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116108 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116111 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:21.120090 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116113 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116116 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116118 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116121 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116123 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116128 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116130 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116133 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116135 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116138 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116141 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116143 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116146 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116148 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116151 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116153 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116156 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116158 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116161 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116163 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116166 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:21.120603 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116168 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116171 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116176 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116179 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116181 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116184 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116186 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116189 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116192 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116194 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116197 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116199 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116202 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116204 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116207 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116209 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116213 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116216 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116218 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116221 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:21.121442 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116288 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116295 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116302 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116306 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116312 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116316 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116322 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116328 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116333 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116337 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116341 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116344 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116347 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116350 2568 flags.go:64] FLAG: --cgroup-root="" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116354 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116357 2568 flags.go:64] FLAG: --client-ca-file="" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116360 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116363 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116367 2568 flags.go:64] FLAG: --cluster-domain="" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116370 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116373 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116376 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116379 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116383 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 10:32:21.121977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116387 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116390 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116393 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116396 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116400 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116403 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116407 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116409 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116414 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116417 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116420 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116423 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116426 2568 flags.go:64] FLAG: --enable-server="true" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116429 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116434 2568 flags.go:64] FLAG: --event-burst="100" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116437 2568 flags.go:64] FLAG: --event-qps="50" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116440 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116442 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116445 2568 flags.go:64] FLAG: --eviction-hard="" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116449 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116452 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116455 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116459 2568 flags.go:64] FLAG: --eviction-soft="" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116462 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116464 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 10:32:21.122677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116467 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116470 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116473 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116476 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116479 2568 flags.go:64] FLAG: --feature-gates="" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116482 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116485 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116488 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116491 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116494 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116497 2568 flags.go:64] FLAG: --help="false" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116500 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116504 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116506 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116510 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116513 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116517 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116520 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116523 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116526 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116529 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116532 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116535 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116537 2568 flags.go:64] FLAG: --kube-reserved="" Jul 16 10:32:21.123967 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116540 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116543 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116546 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116549 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116552 2568 flags.go:64] FLAG: --lock-file="" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116556 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116559 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116562 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116568 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116570 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116573 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116576 2568 flags.go:64] FLAG: --logging-format="text" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116579 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116582 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116585 2568 flags.go:64] FLAG: --manifest-url="" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116588 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116592 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116595 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116599 2568 flags.go:64] FLAG: --max-pods="110" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116602 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116605 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116608 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116611 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116614 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116617 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 10:32:21.125046 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116620 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116626 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116629 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116633 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116636 2568 flags.go:64] FLAG: --pod-cidr="" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116639 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116644 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116647 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116650 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116653 2568 flags.go:64] FLAG: --port="10250" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116656 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116659 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b46e151d69ddec6c" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116662 2568 flags.go:64] FLAG: --qos-reserved="" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116666 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116669 2568 flags.go:64] FLAG: --register-node="true" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116672 2568 flags.go:64] FLAG: --register-with-taints="" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116675 2568 flags.go:64] FLAG: --registry-burst="10" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116678 2568 flags.go:64] FLAG: --registry-qps="5" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116681 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116683 2568 flags.go:64] FLAG: --reserved-memory="" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116687 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116690 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116693 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116696 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116698 2568 flags.go:64] FLAG: --runonce="false" Jul 16 10:32:21.126090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116701 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116704 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116707 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116710 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116713 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116716 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116719 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116722 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116725 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116728 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116730 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116734 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116737 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116740 2568 flags.go:64] FLAG: --system-cgroups="" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116742 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116747 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116750 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116753 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116757 2568 flags.go:64] FLAG: --tls-min-version="" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116759 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116765 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116767 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116770 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116773 2568 flags.go:64] FLAG: --v="2" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116777 2568 flags.go:64] FLAG: --version="false" Jul 16 10:32:21.126736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116780 2568 flags.go:64] FLAG: --vmodule="" Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116784 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.116787 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116935 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116940 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116944 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116947 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116950 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116953 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116956 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116959 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116962 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116965 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116967 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116970 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116973 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116976 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116979 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116981 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116985 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:21.127476 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116988 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116991 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116993 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116996 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.116999 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117002 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117005 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117011 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117014 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117017 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117020 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117022 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117025 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117028 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117030 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117033 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117035 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117038 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117041 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117043 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:21.128060 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117046 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117048 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117051 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117053 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117056 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117058 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117061 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117064 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117066 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117069 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117071 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117074 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117077 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117080 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117082 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117085 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117087 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117090 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117092 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117097 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:21.128572 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117099 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117102 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117104 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117108 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117110 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117113 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117116 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117118 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117121 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117123 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117126 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117129 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117132 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117134 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117137 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117141 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117145 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117148 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117150 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117153 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:21.129104 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117156 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117158 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117161 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117163 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117166 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117169 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117172 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117174 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117177 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117179 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117182 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117187 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117191 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117195 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117197 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117200 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117204 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117207 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117210 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:21.129626 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117213 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117216 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.117218 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.117772 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.126189 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.126205 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.126873 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.126885 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126950 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126956 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126959 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126963 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126967 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126970 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126973 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:21.130137 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.126976 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127005 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127009 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127013 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127017 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127020 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127023 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127027 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127029 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127032 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127036 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127038 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127041 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127044 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127047 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127050 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127053 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127056 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127058 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127061 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:21.130542 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127064 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127068 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127071 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127075 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127077 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127080 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127083 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127085 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127088 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127091 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127094 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127097 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127099 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127102 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127105 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127108 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127111 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127114 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127117 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127119 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:21.131196 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127122 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127125 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127128 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127130 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127133 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127136 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127139 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127141 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127144 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127147 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127149 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127152 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127155 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127158 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127161 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127163 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127166 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127169 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127171 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127174 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:21.131718 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127177 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127179 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127182 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127185 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127187 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127190 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127193 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127195 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127198 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127201 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127203 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127206 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127211 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127215 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127219 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127222 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127225 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127228 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127231 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:21.132264 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127234 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127237 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127240 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127242 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127247 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127250 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127254 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127257 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127260 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127263 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127266 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127269 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127272 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.127278 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127406 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:21.132777 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127412 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127416 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127419 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127422 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127425 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127427 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127430 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127433 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127436 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127438 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127441 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127444 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127447 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127450 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127454 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127456 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127459 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127462 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127465 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127468 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:21.133207 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127470 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127473 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127476 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127479 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127481 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127484 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127487 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127490 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127493 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127496 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127499 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127502 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127504 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127507 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127510 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127513 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127516 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127518 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127521 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127524 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:21.133739 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127526 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127529 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127532 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127535 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127539 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127544 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127547 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127550 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127554 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127557 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127560 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127562 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127565 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127568 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127570 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127573 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127576 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127579 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127581 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:21.134267 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127584 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127587 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127590 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127592 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127595 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127598 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127600 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127603 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127606 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127608 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127611 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127615 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127618 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127621 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127624 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127627 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127629 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127632 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127640 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127644 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:21.134766 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127648 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127651 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127654 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127657 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127659 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127662 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127666 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127669 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127671 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127674 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127677 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127680 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127683 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127685 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127688 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127691 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127694 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127696 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:21.135299 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:21.127699 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.127704 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.127822 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.130694 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.131886 2568 server.go:1019] "Starting client certificate rotation" Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.132004 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.132047 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 10:32:21.135752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.134804 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 10:32:21.138861 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.138844 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 10:32:21.152336 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.152315 2568 log.go:25] "Validated CRI v1 runtime API" Jul 16 10:32:21.154432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.154414 2568 log.go:25] "Validated CRI v1 image API" Jul 16 10:32:21.155306 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.155275 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 10:32:21.160853 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.160834 2568 fs.go:135] Filesystem UUIDs: map[0dcc550e-765b-4880-b5a6-a5d59f3f775a:/dev/nvme0n1p4 31d69278-552b-4c3d-a099-48a6a337d778:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 16 10:32:21.160904 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.160854 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 10:32:21.167587 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.167422 2568 manager.go:216] Machine: {Timestamp:2026-07-16 10:32:21.165258371 +0000 UTC m=+0.301985037 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100127 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fdea24e44f489a1d874f6697cd17c SystemUUID:ec2fdea2-4e44-f489-a1d8-74f6697cd17c BootID:8f31e59c-e067-41f6-b031-be638812cc3b Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:af:e7:cd:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:af:e7:cd:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:68:07:61:7d:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 10:32:21.167587 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.167587 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 10:32:21.167700 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.167673 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 10:32:21.170214 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.170186 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 10:32:21.170351 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.170219 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 10:32:21.170444 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.170427 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 10:32:21.171314 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.171303 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 10:32:21.171349 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.171317 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 10:32:21.171349 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.171324 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 10:32:21.171939 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.171928 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 10:32:21.173004 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.172976 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 10:32:21.175453 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175441 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 16 10:32:21.175579 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175570 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 10:32:21.175611 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175599 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 16 10:32:21.175644 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175612 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 10:32:21.175644 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175627 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 10:32:21.175644 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175636 2568 kubelet.go:398] "Adding apiserver pod source" Jul 16 10:32:21.175733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.175645 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 10:32:21.178313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.178289 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 10:32:21.178448 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.178346 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 10:32:21.180651 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.180635 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 10:32:21.182270 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182254 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 10:32:21.182322 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182289 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 10:32:21.182503 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182491 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 10:32:21.182548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182508 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 10:32:21.182548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182519 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 10:32:21.182548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182525 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 10:32:21.182548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182536 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 10:32:21.182548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182546 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 10:32:21.182950 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182942 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 10:32:21.182950 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182950 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 10:32:21.183041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182957 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 10:32:21.183041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182963 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 10:32:21.183041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.182971 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 10:32:21.183041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.183000 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 10:32:21.183776 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.183764 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 10:32:21.183810 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.183778 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 10:32:21.187161 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.187146 2568 server.go:1268] "Started kubelet" Jul 16 10:32:21.187305 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.187220 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 10:32:21.187374 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.187284 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 10:32:21.187425 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.187400 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 10:32:21.187936 ip-10-0-138-186 systemd[1]: Started Kubernetes Kubelet. Jul 16 10:32:21.188472 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.188451 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 10:32:21.189130 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.189117 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 10:32:21.189654 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.189628 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 10:32:21.189714 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.189632 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 10:32:21.189748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.189730 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 16 10:32:21.192950 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.192049 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-186.ec2.internal.18c2be930d710095 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-186.ec2.internal,UID:ip-10-0-138-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-186.ec2.internal,},FirstTimestamp:2026-07-16 10:32:21.187117205 +0000 UTC m=+0.323843872,LastTimestamp:2026-07-16 10:32:21.187117205 +0000 UTC m=+0.323843872,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-186.ec2.internal,}" Jul 16 10:32:21.193840 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.193820 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 10:32:21.193916 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.193836 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 10:32:21.194074 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.194056 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.194592 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194571 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 10:32:21.194692 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194597 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 10:32:21.194692 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194610 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 10:32:21.194807 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194704 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 16 10:32:21.194807 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194716 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 16 10:32:21.194973 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194954 2568 factory.go:55] Registering systemd factory Jul 16 10:32:21.194973 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.194974 2568 factory.go:223] Registration of the systemd container factory successfully Jul 16 10:32:21.195257 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.195239 2568 factory.go:153] Registering CRI-O factory Jul 16 10:32:21.195327 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.195261 2568 factory.go:223] Registration of the crio container factory successfully Jul 16 10:32:21.195327 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.195316 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 10:32:21.195436 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.195361 2568 factory.go:103] Registering Raw factory Jul 16 10:32:21.195436 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.195376 2568 manager.go:1196] Started watching for new ooms in manager Jul 16 10:32:21.195906 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.195884 2568 manager.go:349] Starting recovery of all containers Jul 16 10:32:21.196243 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.196213 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 10:32:21.196816 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.196782 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 10:32:21.196919 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.196900 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 10:32:21.208298 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.208281 2568 manager.go:354] Recovery completed Jul 16 10:32:21.213710 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.213696 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.215699 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.215671 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.215756 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.215719 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.215756 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.215734 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.216426 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.216406 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 10:32:21.216426 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.216421 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.216585 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.216422 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 10:32:21.216585 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.216496 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 16 10:32:21.217267 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.217248 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.217363 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.217281 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.217363 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.217297 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.217767 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.217750 2568 policy_none.go:49] "None policy: Start" Jul 16 10:32:21.217767 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.217697 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-186.ec2.internal.18c2be930f2528fa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-186.ec2.internal,UID:ip-10-0-138-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-186.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-186.ec2.internal,},FirstTimestamp:2026-07-16 10:32:21.215701242 +0000 UTC m=+0.352427913,LastTimestamp:2026-07-16 10:32:21.215701242 +0000 UTC m=+0.352427913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-186.ec2.internal,}" Jul 16 10:32:21.217767 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.217770 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 10:32:21.217977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.217782 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 10:32:21.218936 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.218923 2568 policy_none.go:47] "Start" Jul 16 10:32:21.226893 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.226874 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l8ncl" Jul 16 10:32:21.229128 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.229064 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-186.ec2.internal.18c2be930f258b5b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-186.ec2.internal,UID:ip-10-0-138-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-186.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-186.ec2.internal,},FirstTimestamp:2026-07-16 10:32:21.215726427 +0000 UTC m=+0.352453096,LastTimestamp:2026-07-16 10:32:21.215726427 +0000 UTC m=+0.352453096,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-186.ec2.internal,}" Jul 16 10:32:21.236851 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.236736 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l8ncl" Jul 16 10:32:21.238568 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.238498 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-186.ec2.internal.18c2be930f25c565 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-186.ec2.internal,UID:ip-10-0-138-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-186.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-186.ec2.internal,},FirstTimestamp:2026-07-16 10:32:21.215741285 +0000 UTC m=+0.352467954,LastTimestamp:2026-07-16 10:32:21.215741285 +0000 UTC m=+0.352467954,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-186.ec2.internal,}" Jul 16 10:32:21.260593 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.260577 2568 manager.go:337] "Starting Device Plugin manager" Jul 16 10:32:21.260689 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.260615 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 10:32:21.260689 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.260629 2568 server.go:87] "Starting device plugin registration server" Jul 16 10:32:21.260878 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.260865 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 10:32:21.260954 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.260880 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 10:32:21.261021 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.260969 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 10:32:21.261074 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.261061 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 10:32:21.261120 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.261090 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 10:32:21.261620 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.261600 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 10:32:21.261688 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.261646 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.363581 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.363520 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.365562 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.364789 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.365562 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.364823 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.365562 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.364839 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.365562 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.364873 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.374755 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.374733 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.374853 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.374758 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-186.ec2.internal\": node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.379902 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.379866 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 10:32:21.381167 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.381147 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 10:32:21.381247 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.381181 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 10:32:21.381247 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.381201 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 10:32:21.381247 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.381230 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 10:32:21.383687 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.383664 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:21.407041 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.407019 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.481473 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.481435 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal"] Jul 16 10:32:21.481564 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.481510 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.482457 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.482441 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.482528 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.482472 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.482528 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.482482 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.484856 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.484843 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.485000 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.484983 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.485080 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.485027 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.486013 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.485996 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.486103 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.486028 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.486103 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.486044 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.486213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.486027 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.486213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.486188 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.486213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.486203 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.488297 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.488278 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.488387 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.488313 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:21.489126 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.489109 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:21.489204 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.489139 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:21.489204 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.489150 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:21.495755 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.495737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cbc50c4153caf2739fb4e20e06dc6519-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal\" (UID: \"cbc50c4153caf2739fb4e20e06dc6519\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.495846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.495769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbc50c4153caf2739fb4e20e06dc6519-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal\" (UID: \"cbc50c4153caf2739fb4e20e06dc6519\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.495846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.495795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a99e495f424a8870814d989f4c4e9e7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-186.ec2.internal\" (UID: \"3a99e495f424a8870814d989f4c4e9e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.507090 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.507069 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.513776 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.513761 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-186.ec2.internal\" not found" node="ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.518962 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.518946 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-186.ec2.internal\" not found" node="ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.595970 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.595946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbc50c4153caf2739fb4e20e06dc6519-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal\" (UID: \"cbc50c4153caf2739fb4e20e06dc6519\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.596044 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.595979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a99e495f424a8870814d989f4c4e9e7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-186.ec2.internal\" (UID: \"3a99e495f424a8870814d989f4c4e9e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.596044 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.595998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cbc50c4153caf2739fb4e20e06dc6519-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal\" (UID: \"cbc50c4153caf2739fb4e20e06dc6519\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.596123 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.596059 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbc50c4153caf2739fb4e20e06dc6519-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal\" (UID: \"cbc50c4153caf2739fb4e20e06dc6519\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.596123 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.596073 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a99e495f424a8870814d989f4c4e9e7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-186.ec2.internal\" (UID: \"3a99e495f424a8870814d989f4c4e9e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.596123 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.596058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cbc50c4153caf2739fb4e20e06dc6519-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal\" (UID: \"cbc50c4153caf2739fb4e20e06dc6519\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.607880 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.607862 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.708397 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.708338 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.808549 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.808516 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:21.815138 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.815116 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.821342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:21.821322 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" Jul 16 10:32:21.908941 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:21.908880 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:22.009373 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.009295 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-186.ec2.internal\" not found" Jul 16 10:32:22.056804 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.056780 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:22.095704 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.095669 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" Jul 16 10:32:22.107681 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.107661 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 10:32:22.109592 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.109577 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" Jul 16 10:32:22.125545 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.125525 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 10:32:22.132185 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.132166 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 10:32:22.132312 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.132292 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 10:32:22.132367 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.132310 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 10:32:22.176041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.176017 2568 apiserver.go:52] "Watching apiserver" Jul 16 10:32:22.194381 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.194360 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 10:32:22.194499 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.194361 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 10:32:22.194746 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.194723 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv","openshift-cluster-node-tuning-operator/tuned-b87ts","openshift-dns/node-resolver-tds2p","openshift-image-registry/node-ca-qpkls","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal","openshift-multus/multus-8dt95","kube-system/konnectivity-agent-hkrc4","openshift-multus/multus-additional-cni-plugins-69v7m","openshift-multus/network-metrics-daemon-w67sh","openshift-network-diagnostics/network-check-target-lpgjl","openshift-network-operator/iptables-alerter-bvmgd","openshift-ovn-kubernetes/ovnkube-node-p6m4b","kube-system/global-pull-secret-syncer-hnfmv","kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal"] Jul 16 10:32:22.197376 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.197361 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.199459 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.199442 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.200053 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.200037 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.200192 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.200176 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.200451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.200435 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 10:32:22.200527 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.200472 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c69cb\"" Jul 16 10:32:22.201561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.201541 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.202072 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.202044 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-crxpx\"" Jul 16 10:32:22.202154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.202103 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.202154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.202108 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.203672 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.203656 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.203990 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.203970 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.204134 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.204115 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.204220 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.204128 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fqf46\"" Jul 16 10:32:22.205193 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.205175 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 10:32:22.205886 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.205868 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.206275 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.206253 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.206370 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.206336 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 10:32:22.206433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.206382 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rkxvk\"" Jul 16 10:32:22.206433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.206393 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.207960 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.207943 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.208293 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.208269 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.208390 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.208376 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.208642 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.208626 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 10:32:22.208700 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.208626 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 10:32:22.208700 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.208680 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kjhsp\"" Jul 16 10:32:22.210264 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.210245 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.210716 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.210692 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 10:32:22.211072 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.211049 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jtz2h\"" Jul 16 10:32:22.211188 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.211064 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 10:32:22.213180 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.213155 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hrn99\"" Jul 16 10:32:22.214356 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.213310 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 10:32:22.214356 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.213684 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 10:32:22.214356 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.213796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.214356 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.213885 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:22.216736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.216716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:22.216823 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.216793 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:22.218996 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.218958 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.221720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.221704 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.221810 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.221767 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9vt4\"" Jul 16 10:32:22.221810 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.221784 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.221911 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.221857 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 10:32:22.222143 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.222127 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.224689 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.224663 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.224779 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.224724 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:22.225145 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.225121 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 10:32:22.225828 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.225806 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 10:32:22.225918 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.225894 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 10:32:22.225918 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.225905 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 10:32:22.226065 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.225968 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 10:32:22.226065 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.226026 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 10:32:22.226234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.226216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sn4g7\"" Jul 16 10:32:22.234160 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.234140 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k4t89" Jul 16 10:32:22.241787 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.241769 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k4t89" Jul 16 10:32:22.242354 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.242313 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 10:27:22 +0000 UTC" deadline="2028-04-19 19:33:03.28651347 +0000 UTC" Jul 16 10:32:22.242429 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.242355 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15441h0m41.044162081s" Jul 16 10:32:22.295671 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.295652 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 10:32:22.297634 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/952adbb4-0aa5-411b-9b19-5b747409c0fd-iptables-alerter-script\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.297763 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297647 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-kubelet\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.297763 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-log-socket\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.297763 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297699 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6bf38d4a-6103-4faa-be64-575b6953c111-serviceca\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.297763 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297730 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.297763 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297756 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-socket-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297779 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-env-overrides\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297805 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6brw\" (UniqueName: \"kubernetes.io/projected/918635d9-8500-4e39-94ad-25d15af7bb25-kube-api-access-k6brw\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-conf-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297881 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twqvf\" (UniqueName: \"kubernetes.io/projected/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-kube-api-access-twqvf\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-hostroot\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297935 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-slash\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7db05134-ea74-4a24-9dd4-47237ab1719c-kubelet-config\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297967 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7db05134-ea74-4a24-9dd4-47237ab1719c-dbus\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.297986 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-var-lib-kubelet\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.298020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q7h7\" (UniqueName: \"kubernetes.io/projected/dcc93a23-38c7-4c82-a82a-145238a435ed-kube-api-access-5q7h7\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4219292e-b9f3-48a4-b15f-901158a6aa70-cni-binary-copy\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-daemon-config\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-system-cni-dir\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298180 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysctl-d\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-etc-kubernetes\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-registration-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298306 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-host\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298337 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-959vx\" (UniqueName: \"kubernetes.io/projected/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-kube-api-access-959vx\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-system-cni-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-k8s-cni-cncf-io\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.298463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298462 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7r95\" (UniqueName: \"kubernetes.io/projected/4219292e-b9f3-48a4-b15f-901158a6aa70-kube-api-access-j7r95\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysconfig\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298525 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-systemd-units\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-node-log\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-tmp\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljtcg\" (UniqueName: \"kubernetes.io/projected/952adbb4-0aa5-411b-9b19-5b747409c0fd-kube-api-access-ljtcg\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/918635d9-8500-4e39-94ad-25d15af7bb25-ovn-node-metrics-cert\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-socket-dir-parent\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298720 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67b7facf-2c47-468e-981f-7fe93391c68f-konnectivity-ca\") pod \"konnectivity-agent-hkrc4\" (UID: \"67b7facf-2c47-468e-981f-7fe93391c68f\") " pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298744 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysctl-conf\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298767 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-run\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298789 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-run-netns\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298812 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bf38d4a-6103-4faa-be64-575b6953c111-host\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-cnibin\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-sys-fs\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-systemd\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299030 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc93a23-38c7-4c82-a82a-145238a435ed-tmp-dir\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.298983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-cni-multus\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-cnibin\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-sys\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-var-lib-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299112 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299147 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299173 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-device-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299201 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgtcj\" (UniqueName: \"kubernetes.io/projected/2f354882-83a0-4553-be2e-e110068ab961-kube-api-access-sgtcj\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299220 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-systemd\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-tuned\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-ovn\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299320 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-cni-netd\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299345 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-cni-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-kubelet\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.299678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299408 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-modprobe-d\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299458 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-ovnkube-script-lib\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-os-release\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-netns\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-multus-certs\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299555 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgmkw\" (UniqueName: \"kubernetes.io/projected/a32430fa-4663-484f-8691-e11ed2a1a185-kube-api-access-lgmkw\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/952adbb4-0aa5-411b-9b19-5b747409c0fd-host-slash\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-cni-bin\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299647 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-ovnkube-config\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299728 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dcc93a23-38c7-4c82-a82a-145238a435ed-hosts-file\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299753 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-cni-bin\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299788 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67b7facf-2c47-468e-981f-7fe93391c68f-agent-certs\") pod \"konnectivity-agent-hkrc4\" (UID: \"67b7facf-2c47-468e-981f-7fe93391c68f\") " pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-os-release\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.300228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299861 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-lib-modules\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.300748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299880 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-etc-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.300748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299922 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-run-ovn-kubernetes\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.300748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j7rq\" (UniqueName: \"kubernetes.io/projected/6bf38d4a-6103-4faa-be64-575b6953c111-kube-api-access-7j7rq\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.300748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-cni-binary-copy\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.300748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.299996 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-kubernetes\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.400347 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-cni-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-kubelet\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400385 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-modprobe-d\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-ovnkube-script-lib\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-os-release\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400449 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-netns\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400451 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-kubelet\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400472 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-multus-certs\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-multus-certs\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400540 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-modprobe-d\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400567 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-cni-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgmkw\" (UniqueName: \"kubernetes.io/projected/a32430fa-4663-484f-8691-e11ed2a1a185-kube-api-access-lgmkw\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400588 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-os-release\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/952adbb4-0aa5-411b-9b19-5b747409c0fd-host-slash\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-netns\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400663 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/952adbb4-0aa5-411b-9b19-5b747409c0fd-host-slash\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400659 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-cni-bin\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-cni-bin\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-ovnkube-config\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dcc93a23-38c7-4c82-a82a-145238a435ed-hosts-file\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400813 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-cni-bin\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67b7facf-2c47-468e-981f-7fe93391c68f-agent-certs\") pod \"konnectivity-agent-hkrc4\" (UID: \"67b7facf-2c47-468e-981f-7fe93391c68f\") " pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.400898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400871 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dcc93a23-38c7-4c82-a82a-145238a435ed-hosts-file\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.400949 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-cni-bin\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-os-release\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-lib-modules\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401049 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-ovnkube-script-lib\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-etc-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-run-ovn-kubernetes\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j7rq\" (UniqueName: \"kubernetes.io/projected/6bf38d4a-6103-4faa-be64-575b6953c111-kube-api-access-7j7rq\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401200 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-cni-binary-copy\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-os-release\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401248 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-lib-modules\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401256 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401269 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401281 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-ovnkube-config\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401263 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-etc-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401295 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-run-ovn-kubernetes\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.401802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-kubernetes\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401333 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/952adbb4-0aa5-411b-9b19-5b747409c0fd-iptables-alerter-script\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-kubelet\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401366 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-log-socket\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401274 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-kubernetes\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6bf38d4a-6103-4faa-be64-575b6953c111-serviceca\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-kubelet\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401425 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-log-socket\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-socket-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-env-overrides\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401518 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6brw\" (UniqueName: \"kubernetes.io/projected/918635d9-8500-4e39-94ad-25d15af7bb25-kube-api-access-k6brw\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-conf-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twqvf\" (UniqueName: \"kubernetes.io/projected/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-kube-api-access-twqvf\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-hostroot\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-slash\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7db05134-ea74-4a24-9dd4-47237ab1719c-kubelet-config\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.402597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7db05134-ea74-4a24-9dd4-47237ab1719c-dbus\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-var-lib-kubelet\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q7h7\" (UniqueName: \"kubernetes.io/projected/dcc93a23-38c7-4c82-a82a-145238a435ed-kube-api-access-5q7h7\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4219292e-b9f3-48a4-b15f-901158a6aa70-cni-binary-copy\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-daemon-config\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401813 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-system-cni-dir\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401863 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysctl-d\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-etc-kubernetes\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-registration-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401927 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-cni-binary-copy\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401938 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-host\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402007 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-system-cni-dir\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/952adbb4-0aa5-411b-9b19-5b747409c0fd-iptables-alerter-script\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402044 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-socket-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402028 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-etc-kubernetes\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.403580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402057 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-conf-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401949 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6bf38d4a-6103-4faa-be64-575b6953c111-serviceca\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.401958 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402115 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-var-lib-kubelet\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.402147 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.402243 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:22.90220074 +0000 UTC m=+2.038927393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-959vx\" (UniqueName: \"kubernetes.io/projected/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-kube-api-access-959vx\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402279 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-registration-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-system-cni-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402327 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-system-cni-dir\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-k8s-cni-cncf-io\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402346 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/918635d9-8500-4e39-94ad-25d15af7bb25-env-overrides\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402361 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-run-k8s-cni-cncf-io\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402369 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7r95\" (UniqueName: \"kubernetes.io/projected/4219292e-b9f3-48a4-b15f-901158a6aa70-kube-api-access-j7r95\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysconfig\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402401 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-hostroot\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-slash\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.404482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-host\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-systemd-units\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402465 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7db05134-ea74-4a24-9dd4-47237ab1719c-kubelet-config\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-node-log\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402504 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-node-log\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402499 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402538 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-tmp\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402549 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402566 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljtcg\" (UniqueName: \"kubernetes.io/projected/952adbb4-0aa5-411b-9b19-5b747409c0fd-kube-api-access-ljtcg\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402588 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-systemd-units\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysctl-d\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/918635d9-8500-4e39-94ad-25d15af7bb25-ovn-node-metrics-cert\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402635 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysconfig\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402656 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-socket-dir-parent\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67b7facf-2c47-468e-981f-7fe93391c68f-konnectivity-ca\") pod \"konnectivity-agent-hkrc4\" (UID: \"67b7facf-2c47-468e-981f-7fe93391c68f\") " pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402685 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4219292e-b9f3-48a4-b15f-901158a6aa70-cni-binary-copy\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.405238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402727 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysctl-conf\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-run\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402764 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-socket-dir-parent\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402792 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-run-netns\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bf38d4a-6103-4faa-be64-575b6953c111-host\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-cnibin\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402874 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-sysctl-conf\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-sys-fs\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402895 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4219292e-b9f3-48a4-b15f-901158a6aa70-multus-daemon-config\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-systemd\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc93a23-38c7-4c82-a82a-145238a435ed-tmp-dir\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402955 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bf38d4a-6103-4faa-be64-575b6953c111-host\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.402963 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-cni-multus\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-run\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-cnibin\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403121 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-sys\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403147 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-var-lib-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.405863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403188 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-cnibin\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403197 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-systemd\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403242 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403292 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a32430fa-4663-484f-8691-e11ed2a1a185-cnibin\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.403310 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403324 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-device-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtcj\" (UniqueName: \"kubernetes.io/projected/2f354882-83a0-4553-be2e-e110068ab961-kube-api-access-sgtcj\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.403391 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:32:22.903351404 +0000 UTC m=+2.040078071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403393 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4219292e-b9f3-48a4-b15f-901158a6aa70-host-var-lib-cni-multus\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403409 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67b7facf-2c47-468e-981f-7fe93391c68f-konnectivity-ca\") pod \"konnectivity-agent-hkrc4\" (UID: \"67b7facf-2c47-468e-981f-7fe93391c68f\") " pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-systemd\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-run-netns\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-tuned\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-sys\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403489 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-ovn\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.406548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-device-dir\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403516 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-cni-netd\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-var-lib-openvswitch\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403562 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-run-ovn\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/918635d9-8500-4e39-94ad-25d15af7bb25-host-cni-netd\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2f354882-83a0-4553-be2e-e110068ab961-sys-fs\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-systemd\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403672 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7db05134-ea74-4a24-9dd4-47237ab1719c-dbus\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.403800 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc93a23-38c7-4c82-a82a-145238a435ed-tmp-dir\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.404440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a32430fa-4663-484f-8691-e11ed2a1a185-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.405446 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67b7facf-2c47-468e-981f-7fe93391c68f-agent-certs\") pod \"konnectivity-agent-hkrc4\" (UID: \"67b7facf-2c47-468e-981f-7fe93391c68f\") " pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.405443 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/918635d9-8500-4e39-94ad-25d15af7bb25-ovn-node-metrics-cert\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.405561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-tmp\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.407117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.405684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-etc-tuned\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.408942 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.408919 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:22.409010 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.408964 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:22.409010 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.408979 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:22.409092 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.409041 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:22.909021637 +0000 UTC m=+2.045748296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:22.409606 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.409571 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a99e495f424a8870814d989f4c4e9e7.slice/crio-38ea1f5cb43f071f430af4e4a455321d9635683c0b48e6cdedb94fd9fa7f5dfa WatchSource:0}: Error finding container 38ea1f5cb43f071f430af4e4a455321d9635683c0b48e6cdedb94fd9fa7f5dfa: Status 404 returned error can't find the container with id 38ea1f5cb43f071f430af4e4a455321d9635683c0b48e6cdedb94fd9fa7f5dfa Jul 16 10:32:22.410160 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.410135 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc50c4153caf2739fb4e20e06dc6519.slice/crio-eab0accf7ab3622bf46912df0abcaa32c231e2155ded268935c607d6f19b433f WatchSource:0}: Error finding container eab0accf7ab3622bf46912df0abcaa32c231e2155ded268935c607d6f19b433f: Status 404 returned error can't find the container with id eab0accf7ab3622bf46912df0abcaa32c231e2155ded268935c607d6f19b433f Jul 16 10:32:22.411472 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.411448 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgmkw\" (UniqueName: \"kubernetes.io/projected/a32430fa-4663-484f-8691-e11ed2a1a185-kube-api-access-lgmkw\") pod \"multus-additional-cni-plugins-69v7m\" (UID: \"a32430fa-4663-484f-8691-e11ed2a1a185\") " pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.412406 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.412376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7r95\" (UniqueName: \"kubernetes.io/projected/4219292e-b9f3-48a4-b15f-901158a6aa70-kube-api-access-j7r95\") pod \"multus-8dt95\" (UID: \"4219292e-b9f3-48a4-b15f-901158a6aa70\") " pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.413282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.413239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q7h7\" (UniqueName: \"kubernetes.io/projected/dcc93a23-38c7-4c82-a82a-145238a435ed-kube-api-access-5q7h7\") pod \"node-resolver-tds2p\" (UID: \"dcc93a23-38c7-4c82-a82a-145238a435ed\") " pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.413521 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.413498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljtcg\" (UniqueName: \"kubernetes.io/projected/952adbb4-0aa5-411b-9b19-5b747409c0fd-kube-api-access-ljtcg\") pod \"iptables-alerter-bvmgd\" (UID: \"952adbb4-0aa5-411b-9b19-5b747409c0fd\") " pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.414175 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.414150 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtcj\" (UniqueName: \"kubernetes.io/projected/2f354882-83a0-4553-be2e-e110068ab961-kube-api-access-sgtcj\") pod \"aws-ebs-csi-driver-node-wtgdv\" (UID: \"2f354882-83a0-4553-be2e-e110068ab961\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.414263 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.414236 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j7rq\" (UniqueName: \"kubernetes.io/projected/6bf38d4a-6103-4faa-be64-575b6953c111-kube-api-access-7j7rq\") pod \"node-ca-qpkls\" (UID: \"6bf38d4a-6103-4faa-be64-575b6953c111\") " pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.414869 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.414843 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-959vx\" (UniqueName: \"kubernetes.io/projected/803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c-kube-api-access-959vx\") pod \"tuned-b87ts\" (UID: \"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c\") " pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.415260 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.415235 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twqvf\" (UniqueName: \"kubernetes.io/projected/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-kube-api-access-twqvf\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.416410 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.416392 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 10:32:22.417973 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.417954 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6brw\" (UniqueName: \"kubernetes.io/projected/918635d9-8500-4e39-94ad-25d15af7bb25-kube-api-access-k6brw\") pod \"ovnkube-node-p6m4b\" (UID: \"918635d9-8500-4e39-94ad-25d15af7bb25\") " pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.502564 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.502534 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:22.525506 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.525443 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" Jul 16 10:32:22.536294 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.536273 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f354882_83a0_4553_be2e_e110068ab961.slice/crio-5ff6a019ed5b9965311059e3461197561ad5a6b285e414effac87e1c43af2bf6 WatchSource:0}: Error finding container 5ff6a019ed5b9965311059e3461197561ad5a6b285e414effac87e1c43af2bf6: Status 404 returned error can't find the container with id 5ff6a019ed5b9965311059e3461197561ad5a6b285e414effac87e1c43af2bf6 Jul 16 10:32:22.541711 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.541693 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b87ts" Jul 16 10:32:22.547476 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.547384 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:22.549587 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.549566 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803de7ca_a3a9_4fd5_9a85_a0f60e4fc65c.slice/crio-bb23f5b2e74de4c47197a38501931e3a6659de6db4163c6a57acb1e4c9111ba2 WatchSource:0}: Error finding container bb23f5b2e74de4c47197a38501931e3a6659de6db4163c6a57acb1e4c9111ba2: Status 404 returned error can't find the container with id bb23f5b2e74de4c47197a38501931e3a6659de6db4163c6a57acb1e4c9111ba2 Jul 16 10:32:22.563081 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.563061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tds2p" Jul 16 10:32:22.571012 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.570993 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc93a23_38c7_4c82_a82a_145238a435ed.slice/crio-84c565580a593bee087fa4ad4a6027c35c3d42130795d7bcb0350102c18f7950 WatchSource:0}: Error finding container 84c565580a593bee087fa4ad4a6027c35c3d42130795d7bcb0350102c18f7950: Status 404 returned error can't find the container with id 84c565580a593bee087fa4ad4a6027c35c3d42130795d7bcb0350102c18f7950 Jul 16 10:32:22.576026 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.576010 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qpkls" Jul 16 10:32:22.585373 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.585342 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf38d4a_6103_4faa_be64_575b6953c111.slice/crio-2da4dd52fab67181bf186022b1eba808acb99ea4aa87f156e5d84971f40f5c5a WatchSource:0}: Error finding container 2da4dd52fab67181bf186022b1eba808acb99ea4aa87f156e5d84971f40f5c5a: Status 404 returned error can't find the container with id 2da4dd52fab67181bf186022b1eba808acb99ea4aa87f156e5d84971f40f5c5a Jul 16 10:32:22.588446 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.588429 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8dt95" Jul 16 10:32:22.594575 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.594559 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:22.596298 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.596274 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4219292e_b9f3_48a4_b15f_901158a6aa70.slice/crio-5d20f9c378f09eccda216bd7e76ca23965fba7f2fe399aab08e337aacae39089 WatchSource:0}: Error finding container 5d20f9c378f09eccda216bd7e76ca23965fba7f2fe399aab08e337aacae39089: Status 404 returned error can't find the container with id 5d20f9c378f09eccda216bd7e76ca23965fba7f2fe399aab08e337aacae39089 Jul 16 10:32:22.602741 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.602719 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b7facf_2c47_468e_981f_7fe93391c68f.slice/crio-0d76c870341b3b91a1b0802bf3935bd76e1134cda23ab14e9dba0b3ea91f05c3 WatchSource:0}: Error finding container 0d76c870341b3b91a1b0802bf3935bd76e1134cda23ab14e9dba0b3ea91f05c3: Status 404 returned error can't find the container with id 0d76c870341b3b91a1b0802bf3935bd76e1134cda23ab14e9dba0b3ea91f05c3 Jul 16 10:32:22.617739 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.617716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-69v7m" Jul 16 10:32:22.625708 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.625688 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32430fa_4663_484f_8691_e11ed2a1a185.slice/crio-76c24bf43b4ef3c6ef84f00d12e568c6dc280fced00536644bf65f8f267b698f WatchSource:0}: Error finding container 76c24bf43b4ef3c6ef84f00d12e568c6dc280fced00536644bf65f8f267b698f: Status 404 returned error can't find the container with id 76c24bf43b4ef3c6ef84f00d12e568c6dc280fced00536644bf65f8f267b698f Jul 16 10:32:22.629752 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.629734 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvmgd" Jul 16 10:32:22.634589 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.634572 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:22.638241 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.638217 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952adbb4_0aa5_411b_9b19_5b747409c0fd.slice/crio-189795481bdd9a371051f4a0581cecd233eb78222b34c8f71205d6b18bd136ed WatchSource:0}: Error finding container 189795481bdd9a371051f4a0581cecd233eb78222b34c8f71205d6b18bd136ed: Status 404 returned error can't find the container with id 189795481bdd9a371051f4a0581cecd233eb78222b34c8f71205d6b18bd136ed Jul 16 10:32:22.643145 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:22.643124 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918635d9_8500_4e39_94ad_25d15af7bb25.slice/crio-bd3087ac71faeb0b7e8564bef732003de684a742856bc35d17a60c42d69bd230 WatchSource:0}: Error finding container bd3087ac71faeb0b7e8564bef732003de684a742856bc35d17a60c42d69bd230: Status 404 returned error can't find the container with id bd3087ac71faeb0b7e8564bef732003de684a742856bc35d17a60c42d69bd230 Jul 16 10:32:22.907341 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.907260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:22.907492 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:22.907344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:22.907492 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.907404 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:22.907492 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.907446 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:22.907492 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.907478 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:32:23.90745849 +0000 UTC m=+3.044185155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:22.907492 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:22.907494 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:23.907486938 +0000 UTC m=+3.044213595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:23.008157 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.008118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:23.008329 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.008304 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:23.008399 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.008384 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:23.008476 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.008410 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:23.008476 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.008471 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:24.00845593 +0000 UTC m=+3.145182581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:23.245762 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.245586 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:27:22 +0000 UTC" deadline="2027-12-24 00:13:48.147035277 +0000 UTC" Jul 16 10:32:23.245762 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.245659 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12613h41m24.901380605s" Jul 16 10:32:23.371218 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.371006 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:23.383503 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.383179 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:23.383503 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.383316 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:23.423512 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.423417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" event={"ID":"cbc50c4153caf2739fb4e20e06dc6519","Type":"ContainerStarted","Data":"eab0accf7ab3622bf46912df0abcaa32c231e2155ded268935c607d6f19b433f"} Jul 16 10:32:23.451783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.451742 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" event={"ID":"3a99e495f424a8870814d989f4c4e9e7","Type":"ContainerStarted","Data":"38ea1f5cb43f071f430af4e4a455321d9635683c0b48e6cdedb94fd9fa7f5dfa"} Jul 16 10:32:23.480876 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.480748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"bd3087ac71faeb0b7e8564bef732003de684a742856bc35d17a60c42d69bd230"} Jul 16 10:32:23.500225 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.500141 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvmgd" event={"ID":"952adbb4-0aa5-411b-9b19-5b747409c0fd","Type":"ContainerStarted","Data":"189795481bdd9a371051f4a0581cecd233eb78222b34c8f71205d6b18bd136ed"} Jul 16 10:32:23.515199 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.515166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerStarted","Data":"76c24bf43b4ef3c6ef84f00d12e568c6dc280fced00536644bf65f8f267b698f"} Jul 16 10:32:23.526463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.526380 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tds2p" event={"ID":"dcc93a23-38c7-4c82-a82a-145238a435ed","Type":"ContainerStarted","Data":"84c565580a593bee087fa4ad4a6027c35c3d42130795d7bcb0350102c18f7950"} Jul 16 10:32:23.560393 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.560272 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" event={"ID":"2f354882-83a0-4553-be2e-e110068ab961","Type":"ContainerStarted","Data":"5ff6a019ed5b9965311059e3461197561ad5a6b285e414effac87e1c43af2bf6"} Jul 16 10:32:23.582774 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.582613 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkrc4" event={"ID":"67b7facf-2c47-468e-981f-7fe93391c68f","Type":"ContainerStarted","Data":"0d76c870341b3b91a1b0802bf3935bd76e1134cda23ab14e9dba0b3ea91f05c3"} Jul 16 10:32:23.604452 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.604355 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8dt95" event={"ID":"4219292e-b9f3-48a4-b15f-901158a6aa70","Type":"ContainerStarted","Data":"5d20f9c378f09eccda216bd7e76ca23965fba7f2fe399aab08e337aacae39089"} Jul 16 10:32:23.614524 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.614492 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qpkls" event={"ID":"6bf38d4a-6103-4faa-be64-575b6953c111","Type":"ContainerStarted","Data":"2da4dd52fab67181bf186022b1eba808acb99ea4aa87f156e5d84971f40f5c5a"} Jul 16 10:32:23.647562 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.647529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b87ts" event={"ID":"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c","Type":"ContainerStarted","Data":"bb23f5b2e74de4c47197a38501931e3a6659de6db4163c6a57acb1e4c9111ba2"} Jul 16 10:32:23.917376 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.917288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:23.917559 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:23.917424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:23.917623 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.917576 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:23.917680 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.917640 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:25.91762036 +0000 UTC m=+5.054347030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:23.917750 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.917704 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:23.917750 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:23.917739 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:32:25.917727789 +0000 UTC m=+5.054454454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:24.018174 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:24.018135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:24.018407 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:24.018386 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:24.018482 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:24.018415 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:24.018482 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:24.018430 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:24.018594 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:24.018495 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:26.018477107 +0000 UTC m=+5.155203777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:24.251193 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:24.251097 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:27:22 +0000 UTC" deadline="2028-03-07 08:20:06.768088941 +0000 UTC" Jul 16 10:32:24.251193 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:24.251143 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14397h47m42.516951149s" Jul 16 10:32:24.381853 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:24.381795 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:24.382058 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:24.381941 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:24.382058 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:24.382025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:24.382181 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:24.382116 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:24.523733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:24.523649 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:25.384591 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:25.384541 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:25.385031 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:25.384681 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:25.939656 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:25.939617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:25.939858 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:25.939717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:25.939930 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:25.939873 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:25.940014 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:25.939936 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.939915931 +0000 UTC m=+9.076642587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:25.940402 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:25.940378 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:25.940475 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:25.940454 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.940437848 +0000 UTC m=+9.077164503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:26.040828 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:26.040119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:26.040828 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:26.040344 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:26.040828 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:26.040364 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:26.040828 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:26.040376 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:26.040828 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:26.040433 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:30.040416111 +0000 UTC m=+9.177142765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:26.381777 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:26.381683 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:26.381777 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:26.381711 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:26.382038 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:26.381836 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:26.382038 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:26.381956 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:27.388213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:27.388183 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:27.388749 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:27.388321 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:28.381596 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:28.381554 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:28.381800 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:28.381702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:28.381800 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:28.381738 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:28.381919 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:28.381838 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:29.382420 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:29.382274 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:29.382420 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:29.382420 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:29.974699 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:29.974650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:29.974897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:29.974763 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:29.975007 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:29.974962 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:29.975071 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:29.975052 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:37.975017427 +0000 UTC m=+17.111744094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:29.975382 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:29.975276 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:29.975382 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:29.975347 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:32:37.975327023 +0000 UTC m=+17.112053679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:30.075974 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:30.075919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:30.076179 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:30.076085 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:30.076179 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:30.076102 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:30.076179 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:30.076113 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:30.076179 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:30.076169 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:38.076153982 +0000 UTC m=+17.212880634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:30.382275 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:30.382180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:30.382440 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:30.382322 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:30.382440 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:30.382377 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:30.382890 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:30.382572 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:31.383015 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:31.382915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:31.383539 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:31.383058 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:32.382350 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:32.382277 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:32.382519 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:32.382412 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:32.382885 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:32.382284 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:32.382885 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:32.382852 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:33.382452 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:33.382415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:33.382930 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:33.382537 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:34.381697 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:34.381666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:34.381889 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:34.381709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:34.381889 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:34.381788 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:34.382021 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:34.381985 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:35.381983 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:35.381949 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:35.382300 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:35.382086 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:36.382353 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:36.382140 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:36.382780 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:36.382202 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:36.382780 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:36.382445 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:36.382780 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:36.382551 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:37.381644 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:37.381606 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:37.381810 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:37.381744 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:38.038326 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:38.038288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:38.038811 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:38.038352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:38.038811 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.038458 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:38.038811 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.038479 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:38.038811 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.038534 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:54.038512818 +0000 UTC m=+33.175239491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:38.038811 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.038554 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:32:54.038546783 +0000 UTC m=+33.175273433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:38.139672 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:38.139637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:38.139848 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.139816 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:38.139848 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.139839 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:38.139848 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.139848 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:38.139998 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.139894 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:54.139881486 +0000 UTC m=+33.276608136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:38.381438 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:38.381358 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:38.381597 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.381483 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:38.381597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:38.381538 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:38.381694 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:38.381654 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:39.381839 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:39.381803 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:39.382278 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:39.381950 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:40.382114 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:40.382081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:40.382501 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:40.382214 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:40.382501 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:40.382274 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:40.382501 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:40.382368 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:41.382658 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:41.382627 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:41.383096 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:41.382744 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:42.382176 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:42.382140 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:42.382338 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:42.382288 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:42.382338 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:42.382320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:42.382472 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:42.382445 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:43.382005 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:43.381970 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:43.382383 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:43.382089 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:44.382374 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.382215 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:44.382975 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.382222 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:44.382975 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:44.382446 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:44.382975 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:44.382541 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:44.694788 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.694506 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkrc4" event={"ID":"67b7facf-2c47-468e-981f-7fe93391c68f","Type":"ContainerStarted","Data":"a426164528a1af67f519759f5c175cc2f251dd0c3a37fd6b8c7e8424fd2cbf72"} Jul 16 10:32:44.696185 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.696102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8dt95" event={"ID":"4219292e-b9f3-48a4-b15f-901158a6aa70","Type":"ContainerStarted","Data":"149b484001a7adbddb5229a0aa4b8e28630894722814e595ab78ab04b2762536"} Jul 16 10:32:44.697611 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.697581 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qpkls" event={"ID":"6bf38d4a-6103-4faa-be64-575b6953c111","Type":"ContainerStarted","Data":"78dc274aa90c9f529911ea615808fa67fc9a36c8a652075bf7fd416cf37eb926"} Jul 16 10:32:44.699254 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.699202 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b87ts" event={"ID":"803de7ca-a3a9-4fd5-9a85-a0f60e4fc65c","Type":"ContainerStarted","Data":"f15fae2654aff70d809741d5a922c2010f723f822fccd9dc91200220bda2b6fe"} Jul 16 10:32:44.700956 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.700929 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbc50c4153caf2739fb4e20e06dc6519" containerID="fc76a47700fe6f732f68601f0cb291bd7c89c01f63d3ce2f85b79a09393a7808" exitCode=0 Jul 16 10:32:44.701044 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.701011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" event={"ID":"cbc50c4153caf2739fb4e20e06dc6519","Type":"ContainerDied","Data":"fc76a47700fe6f732f68601f0cb291bd7c89c01f63d3ce2f85b79a09393a7808"} Jul 16 10:32:44.702517 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.702485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" event={"ID":"3a99e495f424a8870814d989f4c4e9e7","Type":"ContainerStarted","Data":"6839efef67332b0f528ec1a587bc94a5d2251b6845b501dea821325c8d2d160e"} Jul 16 10:32:44.705235 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705219 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:32:44.705556 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705540 2568 generic.go:356] "Generic (PLEG): container finished" podID="918635d9-8500-4e39-94ad-25d15af7bb25" containerID="5d370edf79fa905de3dab4cce7ffac3704fad247bc040619fdfec9309783da23" exitCode=1 Jul 16 10:32:44.705623 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705604 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"ce4a414d718645c4082e4731b0b0b158839e25ab70439b9a5cbe30a0160f409a"} Jul 16 10:32:44.705666 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"8e02e1fa2dc9a921bd0bc1ec69da1b169fa36b19df48b4e1dbf2ca0e48077bd8"} Jul 16 10:32:44.705708 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705687 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"f1d2498382161543827d69df4007e4aee47fe5e42e9a2ed5fcee470d79ae85f7"} Jul 16 10:32:44.705739 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705730 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"be5dd5a7353683e3ffa9d4630ca6b6a6ce0b2a0a0d16d19653c300e157693d37"} Jul 16 10:32:44.705782 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705763 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerDied","Data":"5d370edf79fa905de3dab4cce7ffac3704fad247bc040619fdfec9309783da23"} Jul 16 10:32:44.705826 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.705811 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"a401e62343822d53dcbc501cad400e4847dade53386388db846b4c02716d3fe6"} Jul 16 10:32:44.707060 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.707040 2568 generic.go:356] "Generic (PLEG): container finished" podID="a32430fa-4663-484f-8691-e11ed2a1a185" containerID="41a6564ddd469a47a57df905b00eb53431f8581c850bae519212edf40bfa9e2b" exitCode=0 Jul 16 10:32:44.707323 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.707105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerDied","Data":"41a6564ddd469a47a57df905b00eb53431f8581c850bae519212edf40bfa9e2b"} Jul 16 10:32:44.708680 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.708615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tds2p" event={"ID":"dcc93a23-38c7-4c82-a82a-145238a435ed","Type":"ContainerStarted","Data":"fb36ac2b8e0e37bbdd44899c181dda1f39efdfe037cba82846aabc723b037468"} Jul 16 10:32:44.710225 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.710205 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" event={"ID":"2f354882-83a0-4553-be2e-e110068ab961","Type":"ContainerStarted","Data":"9bf68c7f923cc9d2807ca520140981a74cee43af5fb784c480496c89c167601c"} Jul 16 10:32:44.714413 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.714374 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hkrc4" podStartSLOduration=1.667332263 podStartE2EDuration="22.714363611s" totalImagesPullingTime="21.047031348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:44.714225748 +0000 UTC m=+23.850952422" watchObservedRunningTime="2026-07-16 10:32:44.714363611 +0000 UTC m=+23.851090283" Jul 16 10:32:44.782169 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.782127 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b87ts" podStartSLOduration=1.648952594 podStartE2EDuration="22.782109875s" totalImagesPullingTime="21.133157281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:44.759511975 +0000 UTC m=+23.896238651" watchObservedRunningTime="2026-07-16 10:32:44.782109875 +0000 UTC m=+23.918836547" Jul 16 10:32:44.782296 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.782248 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tds2p" podStartSLOduration=1.670873754 podStartE2EDuration="22.782240713s" totalImagesPullingTime="21.111366959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:44.781664492 +0000 UTC m=+23.918391166" watchObservedRunningTime="2026-07-16 10:32:44.782240713 +0000 UTC m=+23.918967385" Jul 16 10:32:44.810199 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.810160 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qpkls" podStartSLOduration=1.71541498 podStartE2EDuration="22.810143031s" totalImagesPullingTime="21.094728051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:44.802661886 +0000 UTC m=+23.939388558" watchObservedRunningTime="2026-07-16 10:32:44.810143031 +0000 UTC m=+23.946869703" Jul 16 10:32:44.937457 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.937412 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-186.ec2.internal" podStartSLOduration=22.937395947 podStartE2EDuration="22.937395947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:44.899901613 +0000 UTC m=+24.036628299" watchObservedRunningTime="2026-07-16 10:32:44.937395947 +0000 UTC m=+24.074122618" Jul 16 10:32:44.937678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:44.937660 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8dt95" podStartSLOduration=1.850072881 podStartE2EDuration="22.937655834s" totalImagesPullingTime="21.087582953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:44.937533791 +0000 UTC m=+24.074260451" watchObservedRunningTime="2026-07-16 10:32:44.937655834 +0000 UTC m=+24.074382507" Jul 16 10:32:45.381783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.381752 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:45.381938 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:45.381877 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:45.662813 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.662776 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 10:32:45.714093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.714058 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" event={"ID":"2f354882-83a0-4553-be2e-e110068ab961","Type":"ContainerStarted","Data":"5105d2a40020ed6b71341521a351076bbefa20b27e271cfaf6b77e61795285ff"} Jul 16 10:32:45.718104 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.716187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" event={"ID":"cbc50c4153caf2739fb4e20e06dc6519","Type":"ContainerStarted","Data":"93a0af93dd0abaf9a2c545319309d8b715737c451f4102c876cd96ad75eeaf23"} Jul 16 10:32:45.721188 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.721162 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvmgd" event={"ID":"952adbb4-0aa5-411b-9b19-5b747409c0fd","Type":"ContainerStarted","Data":"4d59fa630dce416c026e3d840a8d739ad16f168ffad48c339635b8e82664f584"} Jul 16 10:32:45.734475 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.734422 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-186.ec2.internal" podStartSLOduration=23.734410164 podStartE2EDuration="23.734410164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:45.733696267 +0000 UTC m=+24.870422940" watchObservedRunningTime="2026-07-16 10:32:45.734410164 +0000 UTC m=+24.871136835" Jul 16 10:32:45.749422 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:45.749379 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bvmgd" podStartSLOduration=2.737746622 podStartE2EDuration="23.749362081s" totalImagesPullingTime="21.011615459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:45.748824184 +0000 UTC m=+24.885550857" watchObservedRunningTime="2026-07-16 10:32:45.749362081 +0000 UTC m=+24.886088752" Jul 16 10:32:46.275070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:46.274970 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T10:32:45.66279622Z","UUID":"f2f93048-f3ed-4866-97b8-b4252b0c85e6","Handler":null,"Name":"","Endpoint":""} Jul 16 10:32:46.276656 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:46.276632 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 10:32:46.276776 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:46.276669 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 10:32:46.382143 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:46.382113 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:46.382347 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:46.382128 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:46.382347 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:46.382252 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:46.382347 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:46.382307 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:47.381697 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.381667 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:47.382166 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:47.381792 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:47.536579 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.536549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:47.537220 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.537196 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:47.728768 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.728700 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:32:47.729164 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.729138 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"79599ace32acf8dfce06ff2e26fba633f4a25dbb4ee673712a25b82b20b432fa"} Jul 16 10:32:47.731391 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.731325 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" event={"ID":"2f354882-83a0-4553-be2e-e110068ab961","Type":"ContainerStarted","Data":"04e2a4be16f6cf4c4152a58341c3ab2a7fc55b9f1f1345a64b53004ae370f123"} Jul 16 10:32:47.731550 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.731531 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:47.732050 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.732036 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hkrc4" Jul 16 10:32:47.777728 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:47.777679 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtgdv" podStartSLOduration=2.087334754 podStartE2EDuration="25.777662183s" totalImagesPullingTime="23.690327429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.777617829 +0000 UTC m=+26.914344502" watchObservedRunningTime="2026-07-16 10:32:47.777662183 +0000 UTC m=+26.914388858" Jul 16 10:32:48.382082 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:48.382043 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:48.382541 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:48.382170 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:48.382541 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:48.382226 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:48.382541 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:48.382346 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:49.382048 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.381847 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:49.382143 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:49.382088 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:49.738846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.738784 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:32:49.739143 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.739122 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"4f9f7ed90c50eeaa2ad622455d1ba3e31a69368340066cf17d148c3bffcc9d86"} Jul 16 10:32:49.739359 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.739338 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:49.739483 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.739392 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:49.739566 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.739544 2568 scope.go:117] "RemoveContainer" containerID="5d370edf79fa905de3dab4cce7ffac3704fad247bc040619fdfec9309783da23" Jul 16 10:32:49.741078 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.741051 2568 generic.go:356] "Generic (PLEG): container finished" podID="a32430fa-4663-484f-8691-e11ed2a1a185" containerID="611b25b37f19e9169c5a3c25e97cea6e5ab771fa3f4a41dbb0e67c2ebfe4ab98" exitCode=0 Jul 16 10:32:49.741172 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.741138 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerDied","Data":"611b25b37f19e9169c5a3c25e97cea6e5ab771fa3f4a41dbb0e67c2ebfe4ab98"} Jul 16 10:32:49.756547 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:49.756528 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:50.382438 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.382259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:50.382928 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.382303 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:50.382928 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:50.382532 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:50.382928 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:50.382591 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:50.747872 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.747847 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:32:50.748387 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.748356 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" event={"ID":"918635d9-8500-4e39-94ad-25d15af7bb25","Type":"ContainerStarted","Data":"fbc252b4f5eecea7cd3a6d376c4affc6ca7004417a0ebe7ad1b4e8c0fc4e91eb"} Jul 16 10:32:50.748661 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.748552 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:50.763851 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.763831 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:32:50.801161 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:50.801120 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" podStartSLOduration=7.593768412 podStartE2EDuration="28.801102044s" totalImagesPullingTime="21.207333632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:50.799261607 +0000 UTC m=+29.935988295" watchObservedRunningTime="2026-07-16 10:32:50.801102044 +0000 UTC m=+29.937828715" Jul 16 10:32:51.199040 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.199006 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w67sh"] Jul 16 10:32:51.199225 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.199128 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:51.199285 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:51.199246 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:51.202758 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.202725 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpgjl"] Jul 16 10:32:51.202916 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.202847 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:51.202993 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:51.202953 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:51.203439 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.203414 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hnfmv"] Jul 16 10:32:51.203548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.203497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:51.203600 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:51.203569 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:51.755283 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.755253 2568 generic.go:356] "Generic (PLEG): container finished" podID="a32430fa-4663-484f-8691-e11ed2a1a185" containerID="26febd6951ae151bca2ff750f42a9d841ffb0063d79de59a31ae67341a3dd189" exitCode=0 Jul 16 10:32:51.755715 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:51.755329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerDied","Data":"26febd6951ae151bca2ff750f42a9d841ffb0063d79de59a31ae67341a3dd189"} Jul 16 10:32:52.382210 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:52.382160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:52.382404 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:52.382321 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:53.381769 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:53.381741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:53.382276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:53.381741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:53.382276 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:53.381856 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:53.382276 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:53.381923 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:53.762933 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:53.762889 2568 generic.go:356] "Generic (PLEG): container finished" podID="a32430fa-4663-484f-8691-e11ed2a1a185" containerID="e60b47eed9432f4926381ebe1e163ccab808adbadd778171895c1537d8c6f140" exitCode=0 Jul 16 10:32:53.763065 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:53.762963 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerDied","Data":"e60b47eed9432f4926381ebe1e163ccab808adbadd778171895c1537d8c6f140"} Jul 16 10:32:54.054492 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:54.054414 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:54.054492 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:54.054469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:54.054663 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.054561 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:54.054663 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.054563 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:54.054663 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.054616 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs podName:3dbe7c71-4f77-4fb2-ae63-8ce1af82e903 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:26.054599496 +0000 UTC m=+65.191326163 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs") pod "network-metrics-daemon-w67sh" (UID: "3dbe7c71-4f77-4fb2-ae63-8ce1af82e903") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:54.054663 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.054630 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret podName:7db05134-ea74-4a24-9dd4-47237ab1719c nodeName:}" failed. No retries permitted until 2026-07-16 10:33:26.054623849 +0000 UTC m=+65.191350498 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret") pod "global-pull-secret-syncer-hnfmv" (UID: "7db05134-ea74-4a24-9dd4-47237ab1719c") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:54.155150 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:54.155122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:54.155286 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.155245 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:54.155286 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.155259 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:54.155286 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.155268 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ggdjf for pod openshift-network-diagnostics/network-check-target-lpgjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:54.155398 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.155313 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf podName:3f851813-5ea5-4cb7-9a4a-4fb2180e239f nodeName:}" failed. No retries permitted until 2026-07-16 10:33:26.155300523 +0000 UTC m=+65.292027186 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggdjf" (UniqueName: "kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf") pod "network-check-target-lpgjl" (UID: "3f851813-5ea5-4cb7-9a4a-4fb2180e239f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:54.381574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:54.381542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:54.381721 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:54.381667 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:55.381833 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:55.381806 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:55.382280 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:55.382041 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:55.382280 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:55.381993 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:55.382280 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:55.382157 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:56.381785 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:56.381755 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:56.381954 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:56.381854 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpgjl" podUID="3f851813-5ea5-4cb7-9a4a-4fb2180e239f" Jul 16 10:32:57.382589 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.382400 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:57.383040 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.382472 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:57.383040 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.382702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hnfmv" podUID="7db05134-ea74-4a24-9dd4-47237ab1719c" Jul 16 10:32:57.383040 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.382791 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w67sh" podUID="3dbe7c71-4f77-4fb2-ae63-8ce1af82e903" Jul 16 10:32:57.530924 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.530832 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-186.ec2.internal" event="NodeReady" Jul 16 10:32:57.531079 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.530997 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 10:32:57.599395 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.599364 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c56d5d8dc-r4tvc"] Jul 16 10:32:57.601710 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.601690 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.603861 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.603836 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9"] Jul 16 10:32:57.605433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.605413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.606825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.606796 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 10:32:57.606939 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.606921 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j5q6\"" Jul 16 10:32:57.606939 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.606894 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 10:32:57.619464 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.619441 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.620892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.620865 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-ttkps"] Jul 16 10:32:57.622418 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.622392 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 10:32:57.622527 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.622440 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 10:32:57.622527 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.622459 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 10:32:57.622527 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.622459 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5wqfg\"" Jul 16 10:32:57.622676 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.622394 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.623243 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.623227 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.635474 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.635457 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.635675 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.635537 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8fbnf\"" Jul 16 10:32:57.635770 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.635715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 10:32:57.636077 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.635950 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.636077 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.635959 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 10:32:57.637299 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.637094 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln"] Jul 16 10:32:57.637987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.637968 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 10:32:57.639805 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.639783 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz"] Jul 16 10:32:57.639896 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.639864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.641702 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.641682 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk"] Jul 16 10:32:57.641832 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.641818 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" Jul 16 10:32:57.644078 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.644059 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9"] Jul 16 10:32:57.644159 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.644086 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c56d5d8dc-r4tvc"] Jul 16 10:32:57.644159 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.644138 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:57.651280 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.651259 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 10:32:57.656441 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.656423 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.656757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.656722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g8x8b\"" Jul 16 10:32:57.659740 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.659710 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 10:32:57.659859 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.659786 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.659986 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.659968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kgddh\"" Jul 16 10:32:57.661407 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.661352 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-ttkps"] Jul 16 10:32:57.662743 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.662723 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nq95s\"" Jul 16 10:32:57.662743 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.662738 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.663270 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.663248 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.663572 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.663555 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.667543 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.667527 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 10:32:57.667820 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.667805 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.669574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.669558 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk"] Jul 16 10:32:57.670248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.670226 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2fh28"] Jul 16 10:32:57.672716 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.672696 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 10:32:57.672980 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.672959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.675835 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.675817 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 10:32:57.676545 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.676530 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 10:32:57.681766 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.681748 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x6pbd\"" Jul 16 10:32:57.685048 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.685026 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6"] Jul 16 10:32:57.688298 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.688277 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f8555469b-j5x5d"] Jul 16 10:32:57.689682 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.689616 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" Jul 16 10:32:57.690510 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.690494 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66"] Jul 16 10:32:57.690636 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.690620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.692067 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.692050 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-vs4xw"] Jul 16 10:32:57.692214 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.692197 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.693539 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.693522 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kzxwn"] Jul 16 10:32:57.693624 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.693596 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.695430 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.695415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.696874 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.696859 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.697395 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.696890 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 10:32:57.697872 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.697854 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 10:32:57.697872 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.697865 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 10:32:57.698720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698702 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.698868 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698793 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.698987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698869 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.698987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698809 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fwjjv\"" Jul 16 10:32:57.698987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698823 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 10:32:57.698987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698822 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 10:32:57.699182 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.698845 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g2xd4\"" Jul 16 10:32:57.699794 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.699777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 10:32:57.699873 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.699852 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 10:32:57.699873 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.699863 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.699992 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.699871 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 10:32:57.699992 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.699879 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pw4gb\"" Jul 16 10:32:57.700103 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.700018 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 10:32:57.700156 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.700117 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.700207 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.700149 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rq55r\"" Jul 16 10:32:57.700471 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.700456 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.701027 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.701009 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 10:32:57.710323 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.710157 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-274x8\"" Jul 16 10:32:57.711220 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.710328 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.711414 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.711391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln"] Jul 16 10:32:57.715074 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.715057 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 10:32:57.729837 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.729337 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:57.730005 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.729986 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz"] Jul 16 10:32:57.731006 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.730717 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2fh28"] Jul 16 10:32:57.731645 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.731627 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66"] Jul 16 10:32:57.732480 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.732464 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kzxwn"] Jul 16 10:32:57.749282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.749259 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6"] Jul 16 10:32:57.752940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.752898 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-vs4xw"] Jul 16 10:32:57.763892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.763874 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f8555469b-j5x5d"] Jul 16 10:32:57.779923 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.779877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.780035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.779940 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-trusted-ca\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.780035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.779968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.780035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.779994 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.780159 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3738f020-bb71-4cee-a084-f6a45ec10d00-serving-cert\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.780159 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-certificates\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.780159 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.780159 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcwwx\" (UniqueName: \"kubernetes.io/projected/34af818c-77a4-4346-8165-a15195c93008-kube-api-access-qcwwx\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.780316 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780159 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:57.780316 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/34af818c-77a4-4346-8165-a15195c93008-config\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.780316 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw726\" (UniqueName: \"kubernetes.io/projected/0d333f8d-b877-4374-90a4-f78d1e97a74f-kube-api-access-qw726\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.780316 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780287 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6qj6\" (UniqueName: \"kubernetes.io/projected/03527b67-f070-4f64-8ed1-51213054d473-kube-api-access-n6qj6\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.780468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780323 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e11cee50-17ab-4283-a72b-50a08cd6946b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.780468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-tmp\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.780468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-default-certificate\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.780468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rktw\" (UniqueName: \"kubernetes.io/projected/a7efd23e-0880-4116-a8f7-883df2499fa8-kube-api-access-5rktw\") pod \"ingress-canary-kzxwn\" (UID: \"a7efd23e-0880-4116-a8f7-883df2499fa8\") " pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.780468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780420 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/34af818c-77a4-4346-8165-a15195c93008-serving-cert\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.780674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780469 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-image-registry-private-configuration\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.780674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hbmb\" (UniqueName: \"kubernetes.io/projected/3738f020-bb71-4cee-a084-f6a45ec10d00-kube-api-access-9hbmb\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.780674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780541 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-ca-trust-extracted\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.780674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780565 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dwvx\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-kube-api-access-4dwvx\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.780674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780598 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e11cee50-17ab-4283-a72b-50a08cd6946b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.780674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3738f020-bb71-4cee-a084-f6a45ec10d00-config\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.780887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqwz\" (UniqueName: \"kubernetes.io/projected/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-kube-api-access-hlqwz\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.780887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780754 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlkq7\" (UniqueName: \"kubernetes.io/projected/83755a08-204a-4301-ab37-8093d4970aa2-kube-api-access-hlkq7\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:57.780887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780787 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.780887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780814 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-stats-auth\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.780887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-bound-sa-token\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.780887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmw8v\" (UniqueName: \"kubernetes.io/projected/e11cee50-17ab-4283-a72b-50a08cd6946b-kube-api-access-xmw8v\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrwjc\" (UniqueName: \"kubernetes.io/projected/46ff2d4c-d1a9-478e-ac25-5fc4470890d0-kube-api-access-rrwjc\") pod \"volume-data-source-validator-78495597f6-dmswz\" (UID: \"46ff2d4c-d1a9-478e-ac25-5fc4470890d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.780984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-snapshots\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781071 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7efd23e-0880-4116-a8f7-883df2499fa8-cert\") pod \"ingress-canary-kzxwn\" (UID: \"a7efd23e-0880-4116-a8f7-883df2499fa8\") " pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781087 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-installation-pull-secrets\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3738f020-bb71-4cee-a084-f6a45ec10d00-trusted-ca\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781141 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/186d512c-ec3f-4afd-b5fe-27037435710e-kube-api-access-sjz9n\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.781194 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pl6s\" (UniqueName: \"kubernetes.io/projected/1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2-kube-api-access-9pl6s\") pod \"network-check-source-869dcfcdcd-mvgn6\" (UID: \"1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" Jul 16 10:32:57.781570 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781224 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-serving-cert\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.781570 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/03527b67-f070-4f64-8ed1-51213054d473-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.781570 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d333f8d-b877-4374-90a4-f78d1e97a74f-config-volume\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.781570 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.781308 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d333f8d-b877-4374-90a4-f78d1e97a74f-tmp-dir\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.882722 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.882694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw726\" (UniqueName: \"kubernetes.io/projected/0d333f8d-b877-4374-90a4-f78d1e97a74f-kube-api-access-qw726\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.882940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.882728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6qj6\" (UniqueName: \"kubernetes.io/projected/03527b67-f070-4f64-8ed1-51213054d473-kube-api-access-n6qj6\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.882940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.882751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e11cee50-17ab-4283-a72b-50a08cd6946b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.882940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.882779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-tmp\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.883102 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.882998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-default-certificate\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.883102 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883039 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rktw\" (UniqueName: \"kubernetes.io/projected/a7efd23e-0880-4116-a8f7-883df2499fa8-kube-api-access-5rktw\") pod \"ingress-canary-kzxwn\" (UID: \"a7efd23e-0880-4116-a8f7-883df2499fa8\") " pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.883102 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/34af818c-77a4-4346-8165-a15195c93008-serving-cert\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.883102 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-image-registry-private-configuration\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883123 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbmb\" (UniqueName: \"kubernetes.io/projected/3738f020-bb71-4cee-a084-f6a45ec10d00-kube-api-access-9hbmb\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883130 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-tmp\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-ca-trust-extracted\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dwvx\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-kube-api-access-4dwvx\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e11cee50-17ab-4283-a72b-50a08cd6946b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883221 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3738f020-bb71-4cee-a084-f6a45ec10d00-config\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqwz\" (UniqueName: \"kubernetes.io/projected/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-kube-api-access-hlqwz\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.883262 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883253 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkq7\" (UniqueName: \"kubernetes.io/projected/83755a08-204a-4301-ab37-8093d4970aa2-kube-api-access-hlkq7\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:57.883740 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.883740 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-stats-auth\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.883740 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-bound-sa-token\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.883740 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883317 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmw8v\" (UniqueName: \"kubernetes.io/projected/e11cee50-17ab-4283-a72b-50a08cd6946b-kube-api-access-xmw8v\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.883960 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.883797 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:32:58.38377593 +0000 UTC m=+37.520502598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : configmap references non-existent config key: service-ca.crt Jul 16 10:32:57.883960 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-ca-trust-extracted\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.883960 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.883984 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrwjc\" (UniqueName: \"kubernetes.io/projected/46ff2d4c-d1a9-478e-ac25-5fc4470890d0-kube-api-access-rrwjc\") pod \"volume-data-source-validator-78495597f6-dmswz\" (UID: \"46ff2d4c-d1a9-478e-ac25-5fc4470890d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-snapshots\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7efd23e-0880-4116-a8f7-883df2499fa8-cert\") pod \"ingress-canary-kzxwn\" (UID: \"a7efd23e-0880-4116-a8f7-883df2499fa8\") " pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-installation-pull-secrets\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.884099 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:32:57.884112 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.884114 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c56d5d8dc-r4tvc: secret "image-registry-tls" not found Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.884168 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls podName:2f3e0792-5e71-481e-9b6b-db9c8f80d8c5 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:58.384152778 +0000 UTC m=+37.520879442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls") pod "image-registry-c56d5d8dc-r4tvc" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5") : secret "image-registry-tls" not found Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3738f020-bb71-4cee-a084-f6a45ec10d00-trusted-ca\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/186d512c-ec3f-4afd-b5fe-27037435710e-kube-api-access-sjz9n\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pl6s\" (UniqueName: \"kubernetes.io/projected/1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2-kube-api-access-9pl6s\") pod \"network-check-source-869dcfcdcd-mvgn6\" (UID: \"1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884286 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-serving-cert\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884311 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3738f020-bb71-4cee-a084-f6a45ec10d00-config\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/03527b67-f070-4f64-8ed1-51213054d473-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884343 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e11cee50-17ab-4283-a72b-50a08cd6946b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d333f8d-b877-4374-90a4-f78d1e97a74f-config-volume\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d333f8d-b877-4374-90a4-f78d1e97a74f-tmp-dir\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.884432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-trusted-ca\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3738f020-bb71-4cee-a084-f6a45ec10d00-serving-cert\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884576 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-certificates\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcwwx\" (UniqueName: \"kubernetes.io/projected/34af818c-77a4-4346-8165-a15195c93008-kube-api-access-qcwwx\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/34af818c-77a4-4346-8165-a15195c93008-config\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-snapshots\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.884955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.884943 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3738f020-bb71-4cee-a084-f6a45ec10d00-trusted-ca\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.885487 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.885002 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 10:32:57.885487 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.885008 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/03527b67-f070-4f64-8ed1-51213054d473-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.885487 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.885052 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls podName:03527b67-f070-4f64-8ed1-51213054d473 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:58.385038705 +0000 UTC m=+37.521765384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-448ln" (UID: "03527b67-f070-4f64-8ed1-51213054d473") : secret "cluster-monitoring-operator-tls" not found Jul 16 10:32:57.885487 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.885102 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 10:32:57.885487 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.885132 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:32:58.385120377 +0000 UTC m=+37.521847041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : secret "router-metrics-certs-default" not found Jul 16 10:32:57.885487 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.885265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/34af818c-77a4-4346-8165-a15195c93008-config\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.885794 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.885611 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d333f8d-b877-4374-90a4-f78d1e97a74f-config-volume\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.885848 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.885829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d333f8d-b877-4374-90a4-f78d1e97a74f-tmp-dir\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.886483 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.886331 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 10:32:57.886483 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.886392 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls podName:83755a08-204a-4301-ab37-8093d4970aa2 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:58.386377692 +0000 UTC m=+37.523104363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls") pod "cluster-samples-operator-65db858894-htlnk" (UID: "83755a08-204a-4301-ab37-8093d4970aa2") : secret "samples-operator-tls" not found Jul 16 10:32:57.886649 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.886602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.886889 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.886799 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:32:57.886889 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:57.886858 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls podName:0d333f8d-b877-4374-90a4-f78d1e97a74f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:58.386844133 +0000 UTC m=+37.523570798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls") pod "dns-default-2fh28" (UID: "0d333f8d-b877-4374-90a4-f78d1e97a74f") : secret "dns-default-metrics-tls" not found Jul 16 10:32:57.887482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.887441 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-trusted-ca\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.887572 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.887497 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-certificates\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.887819 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.887798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.889488 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.889034 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-default-certificate\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.889674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.889653 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7efd23e-0880-4116-a8f7-883df2499fa8-cert\") pod \"ingress-canary-kzxwn\" (UID: \"a7efd23e-0880-4116-a8f7-883df2499fa8\") " pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.889999 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.889946 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-image-registry-private-configuration\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.890094 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.890047 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3738f020-bb71-4cee-a084-f6a45ec10d00-serving-cert\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.890501 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.890475 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-installation-pull-secrets\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.890597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.890562 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-serving-cert\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.891259 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.891217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/34af818c-77a4-4346-8165-a15195c93008-serving-cert\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.891354 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.891296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-stats-auth\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.891927 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.891883 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e11cee50-17ab-4283-a72b-50a08cd6946b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.892029 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.892003 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6qj6\" (UniqueName: \"kubernetes.io/projected/03527b67-f070-4f64-8ed1-51213054d473-kube-api-access-n6qj6\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:57.892223 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.892205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw726\" (UniqueName: \"kubernetes.io/projected/0d333f8d-b877-4374-90a4-f78d1e97a74f-kube-api-access-qw726\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:57.899704 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.899656 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hbmb\" (UniqueName: \"kubernetes.io/projected/3738f020-bb71-4cee-a084-f6a45ec10d00-kube-api-access-9hbmb\") pod \"console-operator-7866497494-ttkps\" (UID: \"3738f020-bb71-4cee-a084-f6a45ec10d00\") " pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.899806 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.899794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rktw\" (UniqueName: \"kubernetes.io/projected/a7efd23e-0880-4116-a8f7-883df2499fa8-kube-api-access-5rktw\") pod \"ingress-canary-kzxwn\" (UID: \"a7efd23e-0880-4116-a8f7-883df2499fa8\") " pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:57.901419 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.901399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrwjc\" (UniqueName: \"kubernetes.io/projected/46ff2d4c-d1a9-478e-ac25-5fc4470890d0-kube-api-access-rrwjc\") pod \"volume-data-source-validator-78495597f6-dmswz\" (UID: \"46ff2d4c-d1a9-478e-ac25-5fc4470890d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" Jul 16 10:32:57.901841 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.901801 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqwz\" (UniqueName: \"kubernetes.io/projected/707e6bfc-4cad-4b9e-9712-93f5e1970cf1-kube-api-access-hlqwz\") pod \"insights-operator-68d9dd94cf-vs4xw\" (UID: \"707e6bfc-4cad-4b9e-9712-93f5e1970cf1\") " pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:57.901940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.901888 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/186d512c-ec3f-4afd-b5fe-27037435710e-kube-api-access-sjz9n\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:57.902009 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.901952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dwvx\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-kube-api-access-4dwvx\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.902561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.902519 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkq7\" (UniqueName: \"kubernetes.io/projected/83755a08-204a-4301-ab37-8093d4970aa2-kube-api-access-hlkq7\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:57.903251 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.903208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmw8v\" (UniqueName: \"kubernetes.io/projected/e11cee50-17ab-4283-a72b-50a08cd6946b-kube-api-access-xmw8v\") pod \"kube-storage-version-migrator-operator-799cb7f958-v8v66\" (UID: \"e11cee50-17ab-4283-a72b-50a08cd6946b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:57.904163 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.904146 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pl6s\" (UniqueName: \"kubernetes.io/projected/1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2-kube-api-access-9pl6s\") pod \"network-check-source-869dcfcdcd-mvgn6\" (UID: \"1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" Jul 16 10:32:57.904259 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.904195 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-bound-sa-token\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:57.904892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.904872 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcwwx\" (UniqueName: \"kubernetes.io/projected/34af818c-77a4-4346-8165-a15195c93008-kube-api-access-qcwwx\") pod \"service-ca-operator-7695fc6c8c-cqjb9\" (UID: \"34af818c-77a4-4346-8165-a15195c93008\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.919892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.919862 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" Jul 16 10:32:57.933629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.933610 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:32:57.965629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:57.965600 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" Jul 16 10:32:58.001301 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.000781 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" Jul 16 10:32:58.017541 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.017293 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" Jul 16 10:32:58.027571 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.024785 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" Jul 16 10:32:58.034991 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.032153 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kzxwn" Jul 16 10:32:58.167070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.164725 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9"] Jul 16 10:32:58.168621 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.168404 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-ttkps"] Jul 16 10:32:58.185043 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:58.184953 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34af818c_77a4_4346_8165_a15195c93008.slice/crio-9057517a3fc9af2ac5b6d3fd3d9b207b9b2882d1ff39203747cb7971346cdc00 WatchSource:0}: Error finding container 9057517a3fc9af2ac5b6d3fd3d9b207b9b2882d1ff39203747cb7971346cdc00: Status 404 returned error can't find the container with id 9057517a3fc9af2ac5b6d3fd3d9b207b9b2882d1ff39203747cb7971346cdc00 Jul 16 10:32:58.199664 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.198388 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz"] Jul 16 10:32:58.239813 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.239788 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6"] Jul 16 10:32:58.266540 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.266511 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-vs4xw"] Jul 16 10:32:58.271875 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.271846 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66"] Jul 16 10:32:58.278052 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.278031 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kzxwn"] Jul 16 10:32:58.280860 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:58.280838 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode11cee50_17ab_4283_a72b_50a08cd6946b.slice/crio-57af77bccc30987226c79511fc5c978cb277690efefba8a4de94c2e1972ba86e WatchSource:0}: Error finding container 57af77bccc30987226c79511fc5c978cb277690efefba8a4de94c2e1972ba86e: Status 404 returned error can't find the container with id 57af77bccc30987226c79511fc5c978cb277690efefba8a4de94c2e1972ba86e Jul 16 10:32:58.281698 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:58.281678 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7efd23e_0880_4116_a8f7_883df2499fa8.slice/crio-ebf6f1b2ad40d3ff29132a4d1fd10eba4519d07264e109359fd4148203fe063c WatchSource:0}: Error finding container ebf6f1b2ad40d3ff29132a4d1fd10eba4519d07264e109359fd4148203fe063c: Status 404 returned error can't find the container with id ebf6f1b2ad40d3ff29132a4d1fd10eba4519d07264e109359fd4148203fe063c Jul 16 10:32:58.282754 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:58.282729 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c58eb5f_a41f_44f5_b2ad_a7c7a57437b2.slice/crio-93e693f1d3f728083822806cdb320f40c4098c9d7e3debd13400d667d15654eb WatchSource:0}: Error finding container 93e693f1d3f728083822806cdb320f40c4098c9d7e3debd13400d667d15654eb: Status 404 returned error can't find the container with id 93e693f1d3f728083822806cdb320f40c4098c9d7e3debd13400d667d15654eb Jul 16 10:32:58.284039 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:32:58.283795 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707e6bfc_4cad_4b9e_9712_93f5e1970cf1.slice/crio-d9ffa44c71709e508dee85541983404d3beaa7a712c1c0dc82f747393445013f WatchSource:0}: Error finding container d9ffa44c71709e508dee85541983404d3beaa7a712c1c0dc82f747393445013f: Status 404 returned error can't find the container with id d9ffa44c71709e508dee85541983404d3beaa7a712c1c0dc82f747393445013f Jul 16 10:32:58.381611 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.381579 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:32:58.384421 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.384401 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42vxd\"" Jul 16 10:32:58.388428 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.388404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:58.388516 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.388432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:58.388516 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.388451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:58.388582 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388547 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 10:32:58.388582 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388549 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 10:32:58.388635 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388550 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:32:58.388635 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388600 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls podName:83755a08-204a-4301-ab37-8093d4970aa2 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.388583836 +0000 UTC m=+38.525310500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls") pod "cluster-samples-operator-65db858894-htlnk" (UID: "83755a08-204a-4301-ab37-8093d4970aa2") : secret "samples-operator-tls" not found Jul 16 10:32:58.388635 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388622 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls podName:0d333f8d-b877-4374-90a4-f78d1e97a74f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.388607403 +0000 UTC m=+38.525334057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls") pod "dns-default-2fh28" (UID: "0d333f8d-b877-4374-90a4-f78d1e97a74f") : secret "dns-default-metrics-tls" not found Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.388646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.388691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388722 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.388710328 +0000 UTC m=+38.525436989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : configmap references non-existent config key: service-ca.crt Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.388745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388769 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388780 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c56d5d8dc-r4tvc: secret "image-registry-tls" not found Jul 16 10:32:58.388798 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388798 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.388787289 +0000 UTC m=+38.525513939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : secret "router-metrics-certs-default" not found Jul 16 10:32:58.389054 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388813 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls podName:2f3e0792-5e71-481e-9b6b-db9c8f80d8c5 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.388805092 +0000 UTC m=+38.525531742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls") pod "image-registry-c56d5d8dc-r4tvc" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5") : secret "image-registry-tls" not found Jul 16 10:32:58.389054 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388810 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 10:32:58.389054 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:58.388873 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls podName:03527b67-f070-4f64-8ed1-51213054d473 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.388857145 +0000 UTC m=+38.525583819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-448ln" (UID: "03527b67-f070-4f64-8ed1-51213054d473") : secret "cluster-monitoring-operator-tls" not found Jul 16 10:32:58.784079 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.784040 2568 generic.go:356] "Generic (PLEG): container finished" podID="a32430fa-4663-484f-8691-e11ed2a1a185" containerID="cc52c15c525692d9075b89daa1f39b9ef692ceca7b73f2bc605cf25204aeb63b" exitCode=0 Jul 16 10:32:58.784244 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.784122 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerDied","Data":"cc52c15c525692d9075b89daa1f39b9ef692ceca7b73f2bc605cf25204aeb63b"} Jul 16 10:32:58.785581 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.785532 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kzxwn" event={"ID":"a7efd23e-0880-4116-a8f7-883df2499fa8","Type":"ContainerStarted","Data":"ebf6f1b2ad40d3ff29132a4d1fd10eba4519d07264e109359fd4148203fe063c"} Jul 16 10:32:58.786963 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.786938 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" event={"ID":"707e6bfc-4cad-4b9e-9712-93f5e1970cf1","Type":"ContainerStarted","Data":"d9ffa44c71709e508dee85541983404d3beaa7a712c1c0dc82f747393445013f"} Jul 16 10:32:58.789484 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.789460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" event={"ID":"1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2","Type":"ContainerStarted","Data":"93e693f1d3f728083822806cdb320f40c4098c9d7e3debd13400d667d15654eb"} Jul 16 10:32:58.791513 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.791490 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" event={"ID":"46ff2d4c-d1a9-478e-ac25-5fc4470890d0","Type":"ContainerStarted","Data":"3f713bac7b88c85f885a8e3e826f4bb3c26e3637cf2d4afa4bba9edc65fbf00a"} Jul 16 10:32:58.793103 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.793072 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" event={"ID":"34af818c-77a4-4346-8165-a15195c93008","Type":"ContainerStarted","Data":"9057517a3fc9af2ac5b6d3fd3d9b207b9b2882d1ff39203747cb7971346cdc00"} Jul 16 10:32:58.794399 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.794331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" event={"ID":"e11cee50-17ab-4283-a72b-50a08cd6946b","Type":"ContainerStarted","Data":"57af77bccc30987226c79511fc5c978cb277690efefba8a4de94c2e1972ba86e"} Jul 16 10:32:58.795524 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:58.795465 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ttkps" event={"ID":"3738f020-bb71-4cee-a084-f6a45ec10d00","Type":"ContainerStarted","Data":"7c8eda4a05ca8db9158228d404945980f7af4968a9e8124b7debba03697d3608"} Jul 16 10:32:59.383054 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.382445 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:32:59.383054 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.382456 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:32:59.387694 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.386025 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 10:32:59.389307 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.388328 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 10:32:59.389307 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.388570 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxmn5\"" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.397855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.397900 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.397957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.398043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.398070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.398099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398228 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398282 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls podName:83755a08-204a-4301-ab37-8093d4970aa2 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.398264575 +0000 UTC m=+40.534991225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls") pod "cluster-samples-operator-65db858894-htlnk" (UID: "83755a08-204a-4301-ab37-8093d4970aa2") : secret "samples-operator-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398743 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.398725764 +0000 UTC m=+40.535452417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : configmap references non-existent config key: service-ca.crt Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398813 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398826 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c56d5d8dc-r4tvc: secret "image-registry-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398862 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls podName:2f3e0792-5e71-481e-9b6b-db9c8f80d8c5 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.398850687 +0000 UTC m=+40.535577343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls") pod "image-registry-c56d5d8dc-r4tvc" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5") : secret "image-registry-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398934 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.398981 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls podName:03527b67-f070-4f64-8ed1-51213054d473 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.398968768 +0000 UTC m=+40.535695421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-448ln" (UID: "03527b67-f070-4f64-8ed1-51213054d473") : secret "cluster-monitoring-operator-tls" not found Jul 16 10:32:59.399458 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.399092 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:32:59.400262 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.399155 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls podName:0d333f8d-b877-4374-90a4-f78d1e97a74f nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.399143249 +0000 UTC m=+40.535869900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls") pod "dns-default-2fh28" (UID: "0d333f8d-b877-4374-90a4-f78d1e97a74f") : secret "dns-default-metrics-tls" not found Jul 16 10:32:59.400262 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.399249 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 10:32:59.400262 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:32:59.399282 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.399271248 +0000 UTC m=+40.535997915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : secret "router-metrics-certs-default" not found Jul 16 10:32:59.811299 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.811269 2568 generic.go:356] "Generic (PLEG): container finished" podID="a32430fa-4663-484f-8691-e11ed2a1a185" containerID="827c1d480a02f5a05543fe17a3f79a2dfcd91466abcb20aaf470feec096fdfa5" exitCode=0 Jul 16 10:32:59.811405 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:32:59.811366 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerDied","Data":"827c1d480a02f5a05543fe17a3f79a2dfcd91466abcb20aaf470feec096fdfa5"} Jul 16 10:33:00.825041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:00.824787 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69v7m" event={"ID":"a32430fa-4663-484f-8691-e11ed2a1a185","Type":"ContainerStarted","Data":"6d74d14f7d1dc67cd90b92c5ba0a4aa182cf9d6380787a97a5eb0e2b52f1311b"} Jul 16 10:33:00.826470 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:00.826443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" event={"ID":"46ff2d4c-d1a9-478e-ac25-5fc4470890d0","Type":"ContainerStarted","Data":"86461b3ce638741e2e1fbf50652656da9193b551d152c851ac4c4b7b81e77b9f"} Jul 16 10:33:00.860804 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:00.860711 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-69v7m" podStartSLOduration=7.236414431 podStartE2EDuration="38.860691132s" totalImagesPullingTime="31.624276701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:00.859544145 +0000 UTC m=+39.996270819" watchObservedRunningTime="2026-07-16 10:33:00.860691132 +0000 UTC m=+39.997417799" Jul 16 10:33:00.878296 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:00.878039 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-dmswz" podStartSLOduration=6.3915514909999995 podStartE2EDuration="7.878017325s" totalImagesPullingTime="1.486465834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:00.876812064 +0000 UTC m=+40.013538736" watchObservedRunningTime="2026-07-16 10:33:00.878017325 +0000 UTC m=+40.014743999" Jul 16 10:33:01.017192 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.017112 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:01.137098 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.137020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:01.419671 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.419583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:01.419671 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.419632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:01.419671 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.419665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:33:01.419961 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.419726 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:01.419961 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.419759 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:01.419961 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419802 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:05.419779814 +0000 UTC m=+44.556506465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : configmap references non-existent config key: service-ca.crt Jul 16 10:33:01.419961 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419838 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:01.419961 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419893 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls podName:0d333f8d-b877-4374-90a4-f78d1e97a74f nodeName:}" failed. No retries permitted until 2026-07-16 10:33:05.419877686 +0000 UTC m=+44.556604352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls") pod "dns-default-2fh28" (UID: "0d333f8d-b877-4374-90a4-f78d1e97a74f") : secret "dns-default-metrics-tls" not found Jul 16 10:33:01.419961 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.419837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419901 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419837 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.420027 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls podName:83755a08-204a-4301-ab37-8093d4970aa2 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:05.420015458 +0000 UTC m=+44.556742117 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls") pod "cluster-samples-operator-65db858894-htlnk" (UID: "83755a08-204a-4301-ab37-8093d4970aa2") : secret "samples-operator-tls" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419837 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.420050 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:05.420039581 +0000 UTC m=+44.556766245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : secret "router-metrics-certs-default" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.420067 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls podName:03527b67-f070-4f64-8ed1-51213054d473 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:05.420058158 +0000 UTC m=+44.556784811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-448ln" (UID: "03527b67-f070-4f64-8ed1-51213054d473") : secret "cluster-monitoring-operator-tls" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.419927 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.420084 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c56d5d8dc-r4tvc: secret "image-registry-tls" not found Jul 16 10:33:01.420281 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:01.420111 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls podName:2f3e0792-5e71-481e-9b6b-db9c8f80d8c5 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:05.420103812 +0000 UTC m=+44.556830495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls") pod "image-registry-c56d5d8dc-r4tvc" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5") : secret "image-registry-tls" not found Jul 16 10:33:01.499384 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.499331 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:01.608213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:01.608182 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:04.120197 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:04.120143 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:04.221382 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:04.221353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:04.321890 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:04.321839 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:04.402953 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:04.402839 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:05.454086 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.454058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.454097 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454184 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454195 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c56d5d8dc-r4tvc: secret "image-registry-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.454217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454246 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls podName:2f3e0792-5e71-481e-9b6b-db9c8f80d8c5 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:13.454232043 +0000 UTC m=+52.590958693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls") pod "image-registry-c56d5d8dc-r4tvc" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5") : secret "image-registry-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454263 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:13.454257569 +0000 UTC m=+52.590984220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : configmap references non-existent config key: service-ca.crt Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454290 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.454313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454335 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls podName:03527b67-f070-4f64-8ed1-51213054d473 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:13.45432376 +0000 UTC m=+52.591050411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-448ln" (UID: "03527b67-f070-4f64-8ed1-51213054d473") : secret "cluster-monitoring-operator-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.454350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454370 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.454373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454398 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls podName:0d333f8d-b877-4374-90a4-f78d1e97a74f nodeName:}" failed. No retries permitted until 2026-07-16 10:33:13.454389683 +0000 UTC m=+52.591116360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls") pod "dns-default-2fh28" (UID: "0d333f8d-b877-4374-90a4-f78d1e97a74f") : secret "dns-default-metrics-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454420 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 10:33:05.454433 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454443 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls podName:83755a08-204a-4301-ab37-8093d4970aa2 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:13.454435615 +0000 UTC m=+52.591162266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls") pod "cluster-samples-operator-65db858894-htlnk" (UID: "83755a08-204a-4301-ab37-8093d4970aa2") : secret "samples-operator-tls" not found Jul 16 10:33:05.454893 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454468 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 10:33:05.454893 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:05.454527 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:13.45451193 +0000 UTC m=+52.591238590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : secret "router-metrics-certs-default" not found Jul 16 10:33:05.722373 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.722345 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:05.829673 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.829650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:05.846557 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.846500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" event={"ID":"707e6bfc-4cad-4b9e-9712-93f5e1970cf1","Type":"ContainerStarted","Data":"cda1915651ed4e638a2fd1640b96d286938a83d1c9211a534754c2a4a0113fca"} Jul 16 10:33:05.848962 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.848938 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ttkps" event={"ID":"3738f020-bb71-4cee-a084-f6a45ec10d00","Type":"ContainerStarted","Data":"f4206b930e203c05758741eb743e4836d6da7ac63dc1675cb12553ebf39b1859"} Jul 16 10:33:05.876241 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.876132 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" podStartSLOduration=5.43134625 podStartE2EDuration="12.876116195s" totalImagesPullingTime="7.444769945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:05.876097686 +0000 UTC m=+45.012824362" watchObservedRunningTime="2026-07-16 10:33:05.876116195 +0000 UTC m=+45.012842845" Jul 16 10:33:05.942725 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:05.941628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:06.733000 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.732966 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xp9mw"] Jul 16 10:33:06.769367 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.769327 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xp9mw"] Jul 16 10:33:06.769500 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.769473 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.772025 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.771994 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 10:33:06.772141 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.772000 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 10:33:06.772141 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.772085 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nb74\"" Jul 16 10:33:06.797929 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.797893 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:06.854070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.854043 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" event={"ID":"e11cee50-17ab-4283-a72b-50a08cd6946b","Type":"ContainerStarted","Data":"a72e145cb921b14a7c5500beb28bbaf642245dc936c8aab4f798d3b10a761cd4"} Jul 16 10:33:06.855691 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.855674 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/0.log" Jul 16 10:33:06.855782 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.855706 2568 generic.go:356] "Generic (PLEG): container finished" podID="3738f020-bb71-4cee-a084-f6a45ec10d00" containerID="f4206b930e203c05758741eb743e4836d6da7ac63dc1675cb12553ebf39b1859" exitCode=255 Jul 16 10:33:06.855847 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.855786 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ttkps" event={"ID":"3738f020-bb71-4cee-a084-f6a45ec10d00","Type":"ContainerDied","Data":"f4206b930e203c05758741eb743e4836d6da7ac63dc1675cb12553ebf39b1859"} Jul 16 10:33:06.855999 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.855981 2568 scope.go:117] "RemoveContainer" containerID="f4206b930e203c05758741eb743e4836d6da7ac63dc1675cb12553ebf39b1859" Jul 16 10:33:06.857607 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.857576 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kzxwn" event={"ID":"a7efd23e-0880-4116-a8f7-883df2499fa8","Type":"ContainerStarted","Data":"126a19be9e040b6e8d4dbf83fd8e84210b92ca547d975cc3406d71e94e563da6"} Jul 16 10:33:06.859341 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.859316 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" event={"ID":"1c58eb5f-a41f-44f5-b2ad-a7c7a57437b2","Type":"ContainerStarted","Data":"2ec26a853162601dac3a823af053a95a35cb7f18dd827c6c18f098e6ce15f11b"} Jul 16 10:33:06.860819 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.860798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" event={"ID":"34af818c-77a4-4346-8165-a15195c93008","Type":"ContainerStarted","Data":"6342115fe6ba5027614779723df5b273590f0f6620e45e98a6f05af061a20d3c"} Jul 16 10:33:06.867777 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.867757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxhbx\" (UniqueName: \"kubernetes.io/projected/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-api-access-vxhbx\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.867835 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.867819 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.868061 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.868032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.868596 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.868574 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d768ed5-cd3f-4127-abdc-cc533901dfd4-data-volume\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.868689 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.868611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d768ed5-cd3f-4127-abdc-cc533901dfd4-crio-socket\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.875315 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.875272 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" podStartSLOduration=6.426848278 podStartE2EDuration="13.87525859s" totalImagesPullingTime="7.448410312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:06.873775129 +0000 UTC m=+46.010501803" watchObservedRunningTime="2026-07-16 10:33:06.87525859 +0000 UTC m=+46.011985240" Jul 16 10:33:06.891988 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.891941 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mvgn6" podStartSLOduration=6.435696419 podStartE2EDuration="13.891897609s" totalImagesPullingTime="7.45620119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:06.890598105 +0000 UTC m=+46.027324779" watchObservedRunningTime="2026-07-16 10:33:06.891897609 +0000 UTC m=+46.028624359" Jul 16 10:33:06.920192 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.920127 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:06.931313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.931276 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kzxwn" podStartSLOduration=2.484105691 podStartE2EDuration="9.931262994s" totalImagesPullingTime="7.447157303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:06.929622632 +0000 UTC m=+46.066349305" watchObservedRunningTime="2026-07-16 10:33:06.931262994 +0000 UTC m=+46.067989665" Jul 16 10:33:06.948830 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.948776 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" podStartSLOduration=6.40827142 podStartE2EDuration="13.948755243s" totalImagesPullingTime="7.540483823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:06.947303646 +0000 UTC m=+46.084030321" watchObservedRunningTime="2026-07-16 10:33:06.948755243 +0000 UTC m=+46.085481919" Jul 16 10:33:06.970328 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.970222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxhbx\" (UniqueName: \"kubernetes.io/projected/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-api-access-vxhbx\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.970328 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.970307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.970494 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.970360 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.970494 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:06.970392 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:06.970494 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.970395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d768ed5-cd3f-4127-abdc-cc533901dfd4-data-volume\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.970494 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.970440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d768ed5-cd3f-4127-abdc-cc533901dfd4-crio-socket\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.970494 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:06.970455 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.470438955 +0000 UTC m=+46.607165609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : configmap "kube-rbac-proxy" not found Jul 16 10:33:06.970749 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.970572 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d768ed5-cd3f-4127-abdc-cc533901dfd4-crio-socket\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.971333 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:06.970830 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:06.971333 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:06.970887 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.470872484 +0000 UTC m=+46.607599138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:06.971333 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.971156 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d768ed5-cd3f-4127-abdc-cc533901dfd4-data-volume\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:06.990812 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:06.990741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxhbx\" (UniqueName: \"kubernetes.io/projected/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-api-access-vxhbx\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:07.211698 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.211662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:07.311896 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.311808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:07.412819 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.412780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:07.476250 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.476220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:07.476402 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.476259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:07.476402 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:07.476339 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:07.476402 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:07.476399 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.47638416 +0000 UTC m=+47.613110809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : configmap "kube-rbac-proxy" not found Jul 16 10:33:07.476547 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:07.476410 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:07.476547 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:07.476463 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.476446358 +0000 UTC m=+47.613173022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:07.822173 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.822137 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:07.865176 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.865151 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/1.log" Jul 16 10:33:07.865550 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.865537 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/0.log" Jul 16 10:33:07.865604 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.865571 2568 generic.go:356] "Generic (PLEG): container finished" podID="3738f020-bb71-4cee-a084-f6a45ec10d00" containerID="0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31" exitCode=255 Jul 16 10:33:07.865682 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.865666 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ttkps" event={"ID":"3738f020-bb71-4cee-a084-f6a45ec10d00","Type":"ContainerDied","Data":"0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31"} Jul 16 10:33:07.865729 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.865717 2568 scope.go:117] "RemoveContainer" containerID="f4206b930e203c05758741eb743e4836d6da7ac63dc1675cb12553ebf39b1859" Jul 16 10:33:07.865887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.865863 2568 scope.go:117] "RemoveContainer" containerID="0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31" Jul 16 10:33:07.866121 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:07.866093 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-ttkps_openshift-console-operator(3738f020-bb71-4cee-a084-f6a45ec10d00)\"" pod="openshift-console-operator/console-operator-7866497494-ttkps" podUID="3738f020-bb71-4cee-a084-f6a45ec10d00" Jul 16 10:33:07.892360 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.892336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:07.933996 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.933974 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:33:07.934078 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:07.934030 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:33:08.087179 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.087110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:08.193968 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.193933 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:08.487074 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.486986 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:08.487074 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.487031 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:08.487267 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:08.487111 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:08.487267 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:08.487178 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:10.487162912 +0000 UTC m=+49.623889561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : configmap "kube-rbac-proxy" not found Jul 16 10:33:08.487267 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:08.487212 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:08.487387 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:08.487269 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:10.487256649 +0000 UTC m=+49.623983298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:08.870771 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.870747 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/1.log" Jul 16 10:33:08.871152 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.871102 2568 scope.go:117] "RemoveContainer" containerID="0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31" Jul 16 10:33:08.871279 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:08.871258 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-ttkps_openshift-console-operator(3738f020-bb71-4cee-a084-f6a45ec10d00)\"" pod="openshift-console-operator/console-operator-7866497494-ttkps" podUID="3738f020-bb71-4cee-a084-f6a45ec10d00" Jul 16 10:33:08.941139 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:08.941113 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tds2p_dcc93a23-38c7-4c82-a82a-145238a435ed/dns-node-resolver/0.log" Jul 16 10:33:09.750654 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:09.750608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:09.874005 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:09.873980 2568 scope.go:117] "RemoveContainer" containerID="0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31" Jul 16 10:33:09.874371 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:09.874130 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-ttkps_openshift-console-operator(3738f020-bb71-4cee-a084-f6a45ec10d00)\"" pod="openshift-console-operator/console-operator-7866497494-ttkps" podUID="3738f020-bb71-4cee-a084-f6a45ec10d00" Jul 16 10:33:09.938051 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:09.938026 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qpkls_6bf38d4a-6103-4faa-be64-575b6953c111/node-ca/0.log" Jul 16 10:33:10.123791 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.123758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:10.234677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.234649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:10.504556 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.504482 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:10.504556 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.504517 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:10.504732 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:10.504598 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:10.504732 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:10.504677 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:14.504657822 +0000 UTC m=+53.641384505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : configmap "kube-rbac-proxy" not found Jul 16 10:33:10.504732 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:10.504697 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:10.504832 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:10.504754 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:14.504741049 +0000 UTC m=+53.641467702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:10.728462 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.728428 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:10.736850 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.736828 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kzxwn_a7efd23e-0880-4116-a8f7-883df2499fa8/serve-healthcheck-canary/0.log" Jul 16 10:33:10.837233 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.837201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:10.926597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:10.926572 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:13.377887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.377852 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:13.529166 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.529136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:13.529335 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.529173 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:33:13.529335 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.529248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:13.529335 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.529264 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:13.529335 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.529283 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:33:13.529335 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529318 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529339 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c56d5d8dc-r4tvc: secret "image-registry-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529369 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529396 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529404 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529426 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls podName:2f3e0792-5e71-481e-9b6b-db9c8f80d8c5 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:29.529404538 +0000 UTC m=+68.666131208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls") pod "image-registry-c56d5d8dc-r4tvc" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5") : secret "image-registry-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529445 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls podName:0d333f8d-b877-4374-90a4-f78d1e97a74f nodeName:}" failed. No retries permitted until 2026-07-16 10:33:29.529435757 +0000 UTC m=+68.666162421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls") pod "dns-default-2fh28" (UID: "0d333f8d-b877-4374-90a4-f78d1e97a74f") : secret "dns-default-metrics-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529489 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529504 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls podName:83755a08-204a-4301-ab37-8093d4970aa2 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:29.529488942 +0000 UTC m=+68.666215596 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls") pod "cluster-samples-operator-65db858894-htlnk" (UID: "83755a08-204a-4301-ab37-8093d4970aa2") : secret "samples-operator-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529519 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:29.529512926 +0000 UTC m=+68.666239575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : secret "router-metrics-certs-default" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:13.529495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529560 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls podName:03527b67-f070-4f64-8ed1-51213054d473 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:29.529542534 +0000 UTC m=+68.666269187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-448ln" (UID: "03527b67-f070-4f64-8ed1-51213054d473") : secret "cluster-monitoring-operator-tls" not found Jul 16 10:33:13.529581 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:13.529583 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle podName:186d512c-ec3f-4afd-b5fe-27037435710e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:29.529574517 +0000 UTC m=+68.666301166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle") pod "router-default-7f8555469b-j5x5d" (UID: "186d512c-ec3f-4afd-b5fe-27037435710e") : configmap references non-existent config key: service-ca.crt Jul 16 10:33:14.229372 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.229331 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:14.340931 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.340886 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:14.536419 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.536344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:14.536419 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.536380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:14.536776 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:14.536505 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:14.536776 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:14.536560 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls podName:6d768ed5-cd3f-4127-abdc-cc533901dfd4 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:22.53654786 +0000 UTC m=+61.673274510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp9mw" (UID: "6d768ed5-cd3f-4127-abdc-cc533901dfd4") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:14.536871 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.536854 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d768ed5-cd3f-4127-abdc-cc533901dfd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:14.559165 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.559141 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:14.658171 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.658134 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:14.759276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.759240 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:14.868056 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.868028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:14.962555 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:14.962530 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:15.729367 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:15.729322 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:16.120037 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:16.120004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:16.525446 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:16.525312 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:16.934743 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:16.934715 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:17.282160 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:17.282066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:17.517624 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:17.517590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:17.847216 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:17.847184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:17.948436 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:17.948405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:18.323834 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:18.323801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:18.719332 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:18.719262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:18.786184 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:18.786154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:18.936237 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:18.936211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:19.115142 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:19.115113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:19.523437 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:19.523364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:19.622856 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:19.622832 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:19.725242 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:19.725212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:20.111663 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:20.111629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:20.431613 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:20.431533 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:20.532882 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:20.532848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:20.722449 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:20.722376 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:21.122833 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.122803 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:21.383683 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.383623 2568 scope.go:117] "RemoveContainer" containerID="0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31" Jul 16 10:33:21.514282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.514259 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:21.912791 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.912765 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:33:21.913156 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.913140 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/1.log" Jul 16 10:33:21.913254 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.913175 2568 generic.go:356] "Generic (PLEG): container finished" podID="3738f020-bb71-4cee-a084-f6a45ec10d00" containerID="272be47c98c46e8dd3c08448ba3419b0bf7759bf524951371f864464d056151d" exitCode=255 Jul 16 10:33:21.913254 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.913235 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ttkps" event={"ID":"3738f020-bb71-4cee-a084-f6a45ec10d00","Type":"ContainerDied","Data":"272be47c98c46e8dd3c08448ba3419b0bf7759bf524951371f864464d056151d"} Jul 16 10:33:21.913374 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.913324 2568 scope.go:117] "RemoveContainer" containerID="0785183051a7200739a3bc69f31f6ee09a1bf8d83e2cf335bf991773de4b0e31" Jul 16 10:33:21.913601 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:21.913581 2568 scope.go:117] "RemoveContainer" containerID="272be47c98c46e8dd3c08448ba3419b0bf7759bf524951371f864464d056151d" Jul 16 10:33:21.913812 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:21.913792 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-ttkps_openshift-console-operator(3738f020-bb71-4cee-a084-f6a45ec10d00)\"" pod="openshift-console-operator/console-operator-7866497494-ttkps" podUID="3738f020-bb71-4cee-a084-f6a45ec10d00" Jul 16 10:33:22.133089 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.133055 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:22.279206 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.279126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:22.512111 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.512078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:22.605055 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.605029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:22.607217 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.607200 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d768ed5-cd3f-4127-abdc-cc533901dfd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp9mw\" (UID: \"6d768ed5-cd3f-4127-abdc-cc533901dfd4\") " pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:22.683774 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.683751 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nb74\"" Jul 16 10:33:22.690443 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.690425 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xp9mw" Jul 16 10:33:22.758267 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.758231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:22.774466 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.774440 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p6m4b" Jul 16 10:33:22.847224 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.847194 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xp9mw"] Jul 16 10:33:22.851314 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:22.851289 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d768ed5_cd3f_4127_abdc_cc533901dfd4.slice/crio-56b672eb036c7d81698cdf4c46c093145ada01be055281c7e3c1b12b6fa56a48 WatchSource:0}: Error finding container 56b672eb036c7d81698cdf4c46c093145ada01be055281c7e3c1b12b6fa56a48: Status 404 returned error can't find the container with id 56b672eb036c7d81698cdf4c46c093145ada01be055281c7e3c1b12b6fa56a48 Jul 16 10:33:22.918990 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.918969 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:33:22.920009 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:22.919986 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp9mw" event={"ID":"6d768ed5-cd3f-4127-abdc-cc533901dfd4","Type":"ContainerStarted","Data":"56b672eb036c7d81698cdf4c46c093145ada01be055281c7e3c1b12b6fa56a48"} Jul 16 10:33:23.141757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:23.141686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:23.724931 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:23.724879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:24.120282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:24.120255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:24.519257 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:24.519178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:24.721435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:24.721403 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:24.926977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:24.926931 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:25.709862 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:25.709831 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:25.817838 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:25.817804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:26.140472 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.140433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:33:26.140892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.140529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:33:26.145771 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.145745 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 10:33:26.146503 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.146482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 10:33:26.154050 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.154026 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dbe7c71-4f77-4fb2-ae63-8ce1af82e903-metrics-certs\") pod \"network-metrics-daemon-w67sh\" (UID: \"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903\") " pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:33:26.154340 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.154312 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db05134-ea74-4a24-9dd4-47237ab1719c-original-pull-secret\") pod \"global-pull-secret-syncer-hnfmv\" (UID: \"7db05134-ea74-4a24-9dd4-47237ab1719c\") " pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:33:26.242007 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.241972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:33:26.244787 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.244757 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggdjf\" (UniqueName: \"kubernetes.io/projected/3f851813-5ea5-4cb7-9a4a-4fb2180e239f-kube-api-access-ggdjf\") pod \"network-check-target-lpgjl\" (UID: \"3f851813-5ea5-4cb7-9a4a-4fb2180e239f\") " pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:33:26.293163 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.293138 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42vxd\"" Jul 16 10:33:26.300990 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.300969 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:33:26.404450 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.404375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnfmv" Jul 16 10:33:26.418936 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.418896 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxmn5\"" Jul 16 10:33:26.426423 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:26.426401 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w67sh" Jul 16 10:33:27.722944 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.722915 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w67sh"] Jul 16 10:33:27.725664 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:27.725637 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbe7c71_4f77_4fb2_ae63_8ce1af82e903.slice/crio-de58b9ef7a485c1c93f90f4289545b924a4726750888e22a720f8d76f1df2b7f WatchSource:0}: Error finding container de58b9ef7a485c1c93f90f4289545b924a4726750888e22a720f8d76f1df2b7f: Status 404 returned error can't find the container with id de58b9ef7a485c1c93f90f4289545b924a4726750888e22a720f8d76f1df2b7f Jul 16 10:33:27.933977 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.933865 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:33:27.934139 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.933983 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:33:27.934421 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.934401 2568 scope.go:117] "RemoveContainer" containerID="272be47c98c46e8dd3c08448ba3419b0bf7759bf524951371f864464d056151d" Jul 16 10:33:27.934589 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:27.934568 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-ttkps_openshift-console-operator(3738f020-bb71-4cee-a084-f6a45ec10d00)\"" pod="openshift-console-operator/console-operator-7866497494-ttkps" podUID="3738f020-bb71-4cee-a084-f6a45ec10d00" Jul 16 10:33:27.938119 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.938094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w67sh" event={"ID":"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903","Type":"ContainerStarted","Data":"de58b9ef7a485c1c93f90f4289545b924a4726750888e22a720f8d76f1df2b7f"} Jul 16 10:33:27.939479 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.939458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp9mw" event={"ID":"6d768ed5-cd3f-4127-abdc-cc533901dfd4","Type":"ContainerStarted","Data":"66efa6d06455e8b1629b9cc592731c76b62b8db8ccfb274baa749580cd071a05"} Jul 16 10:33:27.947672 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.947648 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpgjl"] Jul 16 10:33:27.948706 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:27.948690 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hnfmv"] Jul 16 10:33:27.950942 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:27.950900 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db05134_ea74_4a24_9dd4_47237ab1719c.slice/crio-d1b70593c0395ca770b1250953773d3d70556185cac9473bbc736af26b1128f1 WatchSource:0}: Error finding container d1b70593c0395ca770b1250953773d3d70556185cac9473bbc736af26b1128f1: Status 404 returned error can't find the container with id d1b70593c0395ca770b1250953773d3d70556185cac9473bbc736af26b1128f1 Jul 16 10:33:27.951568 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:27.951548 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f851813_5ea5_4cb7_9a4a_4fb2180e239f.slice/crio-ac4ac8de3281aad5b67b0a1a7e75f731c14bcdd3edd7fab343e5e5e8cedbecc3 WatchSource:0}: Error finding container ac4ac8de3281aad5b67b0a1a7e75f731c14bcdd3edd7fab343e5e5e8cedbecc3: Status 404 returned error can't find the container with id ac4ac8de3281aad5b67b0a1a7e75f731c14bcdd3edd7fab343e5e5e8cedbecc3 Jul 16 10:33:28.946282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:28.945611 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpgjl" event={"ID":"3f851813-5ea5-4cb7-9a4a-4fb2180e239f","Type":"ContainerStarted","Data":"4db2a0a755842400eb2bc48f4dcbe8dcf414547f2b1dd1b60dad8d42e2005030"} Jul 16 10:33:28.946282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:28.945669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpgjl" event={"ID":"3f851813-5ea5-4cb7-9a4a-4fb2180e239f","Type":"ContainerStarted","Data":"ac4ac8de3281aad5b67b0a1a7e75f731c14bcdd3edd7fab343e5e5e8cedbecc3"} Jul 16 10:33:28.946282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:28.946234 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:33:28.947887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:28.947855 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hnfmv" event={"ID":"7db05134-ea74-4a24-9dd4-47237ab1719c","Type":"ContainerStarted","Data":"d1b70593c0395ca770b1250953773d3d70556185cac9473bbc736af26b1128f1"} Jul 16 10:33:28.967021 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:28.966970 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lpgjl" podStartSLOduration=66.966954 podStartE2EDuration="1m6.966954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:28.966137783 +0000 UTC m=+68.102864458" watchObservedRunningTime="2026-07-16 10:33:28.966954 +0000 UTC m=+68.103680670" Jul 16 10:33:29.574609 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.574580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:29.574775 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.574625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:29.574775 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.574763 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:33:29.574926 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.574858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:29.574926 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.574887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:29.575028 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.574926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:33:29.575633 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.575577 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186d512c-ec3f-4afd-b5fe-27037435710e-service-ca-bundle\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:29.578032 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.578002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d333f8d-b877-4374-90a4-f78d1e97a74f-metrics-tls\") pod \"dns-default-2fh28\" (UID: \"0d333f8d-b877-4374-90a4-f78d1e97a74f\") " pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:29.578204 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.578002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83755a08-204a-4301-ab37-8093d4970aa2-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-htlnk\" (UID: \"83755a08-204a-4301-ab37-8093d4970aa2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:33:29.578204 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.578157 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d512c-ec3f-4afd-b5fe-27037435710e-metrics-certs\") pod \"router-default-7f8555469b-j5x5d\" (UID: \"186d512c-ec3f-4afd-b5fe-27037435710e\") " pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:29.578603 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.578582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03527b67-f070-4f64-8ed1-51213054d473-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-448ln\" (UID: \"03527b67-f070-4f64-8ed1-51213054d473\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:33:29.578672 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.578582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"image-registry-c56d5d8dc-r4tvc\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:29.717317 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.717285 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j5q6\"" Jul 16 10:33:29.724629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.724602 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:29.754415 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.754387 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nq95s\"" Jul 16 10:33:29.762433 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.762403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" Jul 16 10:33:29.762987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.762964 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kgddh\"" Jul 16 10:33:29.770373 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.770352 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" Jul 16 10:33:29.786258 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.786233 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x6pbd\"" Jul 16 10:33:29.793750 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.793728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:29.812453 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.812432 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fwjjv\"" Jul 16 10:33:29.819976 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.819942 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:29.953783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:29.953753 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w67sh" event={"ID":"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903","Type":"ContainerStarted","Data":"88151bf4a838c2d763a57057d934fb06f47052afd0934e9605e7b84daa664ccd"} Jul 16 10:33:32.490720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:32.490681 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:32.627629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:32.627568 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:32.731294 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:32.731263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:33.171336 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:33.171305 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:33.294157 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:33.294128 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:33.794285 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:33.794256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2fh28"] Jul 16 10:33:33.900178 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:33.900147 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d333f8d_b877_4374_90a4_f78d1e97a74f.slice/crio-000cacffc9e00edaed69dc41c8c3c6d33635eed966bbd59dd5352ea883789c34 WatchSource:0}: Error finding container 000cacffc9e00edaed69dc41c8c3c6d33635eed966bbd59dd5352ea883789c34: Status 404 returned error can't find the container with id 000cacffc9e00edaed69dc41c8c3c6d33635eed966bbd59dd5352ea883789c34 Jul 16 10:33:33.968471 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:33.968438 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hnfmv" event={"ID":"7db05134-ea74-4a24-9dd4-47237ab1719c","Type":"ContainerStarted","Data":"af0da2b5866ec2d4c724b0aeab983b27387add2f76209528754d1e74c6d4555f"} Jul 16 10:33:33.969650 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:33.969623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2fh28" event={"ID":"0d333f8d-b877-4374-90a4-f78d1e97a74f","Type":"ContainerStarted","Data":"000cacffc9e00edaed69dc41c8c3c6d33635eed966bbd59dd5352ea883789c34"} Jul 16 10:33:33.971368 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:33.971345 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w67sh" event={"ID":"3dbe7c71-4f77-4fb2-ae63-8ce1af82e903","Type":"ContainerStarted","Data":"557703d9bece5a26dee1843bbbf22a5ecfbedf154c2c1f1600b5be788943303f"} Jul 16 10:33:34.012927 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.012790 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hnfmv" podStartSLOduration=66.39627784300001 podStartE2EDuration="1m12.012772581s" totalImagesPullingTime="5.616494738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:33.99163717 +0000 UTC m=+73.128363839" watchObservedRunningTime="2026-07-16 10:33:34.012772581 +0000 UTC m=+73.149499254" Jul 16 10:33:34.013712 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.013689 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk"] Jul 16 10:33:34.032566 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.032184 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w67sh" podStartSLOduration=70.509416 podStartE2EDuration="1m12.032164767s" totalImagesPullingTime="1.522748767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:34.022581225 +0000 UTC m=+73.159307898" watchObservedRunningTime="2026-07-16 10:33:34.032164767 +0000 UTC m=+73.168891439" Jul 16 10:33:34.032698 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.032646 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln"] Jul 16 10:33:34.035791 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:34.035771 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03527b67_f070_4f64_8ed1_51213054d473.slice/crio-a87a1f69e46bb1253bb394280f7ca4cb683a83f82a546b2e824fb95c3a4c6287 WatchSource:0}: Error finding container a87a1f69e46bb1253bb394280f7ca4cb683a83f82a546b2e824fb95c3a4c6287: Status 404 returned error can't find the container with id a87a1f69e46bb1253bb394280f7ca4cb683a83f82a546b2e824fb95c3a4c6287 Jul 16 10:33:34.040018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.039898 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f8555469b-j5x5d"] Jul 16 10:33:34.041218 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.041194 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c56d5d8dc-r4tvc"] Jul 16 10:33:34.041967 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:34.041941 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3e0792_5e71_481e_9b6b_db9c8f80d8c5.slice/crio-c8c568b1e24504e4ae1d3cbf29575a36dec084fba2fd9532ac94422ff7208006 WatchSource:0}: Error finding container c8c568b1e24504e4ae1d3cbf29575a36dec084fba2fd9532ac94422ff7208006: Status 404 returned error can't find the container with id c8c568b1e24504e4ae1d3cbf29575a36dec084fba2fd9532ac94422ff7208006 Jul 16 10:33:34.043040 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:34.043016 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186d512c_ec3f_4afd_b5fe_27037435710e.slice/crio-6b70f59042593fcddaf8c3a4b4dbb81766ae1e15ca1b49d9baa966054836482f WatchSource:0}: Error finding container 6b70f59042593fcddaf8c3a4b4dbb81766ae1e15ca1b49d9baa966054836482f: Status 404 returned error can't find the container with id 6b70f59042593fcddaf8c3a4b4dbb81766ae1e15ca1b49d9baa966054836482f Jul 16 10:33:34.170135 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.170105 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:34.267817 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.267783 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:34.979631 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.979521 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" event={"ID":"83755a08-204a-4301-ab37-8093d4970aa2","Type":"ContainerStarted","Data":"48e1a8c92cac5cb7102067c81bdc4a02580a0bfac88423668b7159edfdf52992"} Jul 16 10:33:34.983041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.982919 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp9mw" event={"ID":"6d768ed5-cd3f-4127-abdc-cc533901dfd4","Type":"ContainerStarted","Data":"5a16d5245b84411d86367757d67a5de4c980819a47a94aa39eb729e971cd7a30"} Jul 16 10:33:34.986069 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.985566 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" event={"ID":"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5","Type":"ContainerStarted","Data":"371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1"} Jul 16 10:33:34.986069 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.985607 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" event={"ID":"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5","Type":"ContainerStarted","Data":"c8c568b1e24504e4ae1d3cbf29575a36dec084fba2fd9532ac94422ff7208006"} Jul 16 10:33:34.986069 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.985848 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:34.987642 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.987613 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" event={"ID":"03527b67-f070-4f64-8ed1-51213054d473","Type":"ContainerStarted","Data":"a87a1f69e46bb1253bb394280f7ca4cb683a83f82a546b2e824fb95c3a4c6287"} Jul 16 10:33:34.989645 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.989567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f8555469b-j5x5d" event={"ID":"186d512c-ec3f-4afd-b5fe-27037435710e","Type":"ContainerStarted","Data":"7e46b76c5bf2787fc1aeecbcb6160af0ce139a2229b52ac709dccc2156fd1972"} Jul 16 10:33:34.989645 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:34.989614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f8555469b-j5x5d" event={"ID":"186d512c-ec3f-4afd-b5fe-27037435710e","Type":"ContainerStarted","Data":"6b70f59042593fcddaf8c3a4b4dbb81766ae1e15ca1b49d9baa966054836482f"} Jul 16 10:33:35.012243 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:35.012192 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" podStartSLOduration=73.012173559 podStartE2EDuration="1m13.012173559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:35.010739655 +0000 UTC m=+74.147466329" watchObservedRunningTime="2026-07-16 10:33:35.012173559 +0000 UTC m=+74.148900232" Jul 16 10:33:35.033171 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:35.033121 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f8555469b-j5x5d" podStartSLOduration=42.033102504 podStartE2EDuration="42.033102504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:35.031397143 +0000 UTC m=+74.168123839" watchObservedRunningTime="2026-07-16 10:33:35.033102504 +0000 UTC m=+74.169829174" Jul 16 10:33:35.821041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:35.820958 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:35.824131 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:35.824105 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:35.995070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:35.995035 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:35.996543 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:35.996521 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f8555469b-j5x5d" Jul 16 10:33:39.504673 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:39.504645 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:39.664971 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:39.664938 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:39.762708 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:39.762630 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:40.001825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.001788 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:40.012081 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.012049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" event={"ID":"83755a08-204a-4301-ab37-8093d4970aa2","Type":"ContainerStarted","Data":"89e167f86aafea7a27a41d39ccf48b4330433d44b0b9e2ddd2c549e82675cfbd"} Jul 16 10:33:40.012220 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.012106 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" event={"ID":"83755a08-204a-4301-ab37-8093d4970aa2","Type":"ContainerStarted","Data":"09b44183452e1ae8066f2d15f10854e5d796bab03cf34833400caf5657998b7d"} Jul 16 10:33:40.014259 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.014196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp9mw" event={"ID":"6d768ed5-cd3f-4127-abdc-cc533901dfd4","Type":"ContainerStarted","Data":"1a35d0d7205768b8d79698323f5c5e6352c506a357efda43e75634720160f9ce"} Jul 16 10:33:40.015914 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.015888 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2fh28" event={"ID":"0d333f8d-b877-4374-90a4-f78d1e97a74f","Type":"ContainerStarted","Data":"06dc4c1387de5f6995ff858dff516d98137405eaf5ed7a3ce655268f17955cc9"} Jul 16 10:33:40.016016 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.015945 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2fh28" event={"ID":"0d333f8d-b877-4374-90a4-f78d1e97a74f","Type":"ContainerStarted","Data":"f6cbe1a144640febafe96acd872e70e73edfb5befd2fe27a8191a360f6fd45d8"} Jul 16 10:33:40.016016 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.016011 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:40.017483 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.017462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" event={"ID":"03527b67-f070-4f64-8ed1-51213054d473","Type":"ContainerStarted","Data":"8eb7fa18f3328dae041455e1e1131f562d2e15bb58049e2347e6c19a26ead9dd"} Jul 16 10:33:40.037500 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.037459 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-htlnk" podStartSLOduration=42.036057315 podStartE2EDuration="47.037445609s" totalImagesPullingTime="5.001388294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:40.035255913 +0000 UTC m=+79.171982586" watchObservedRunningTime="2026-07-16 10:33:40.037445609 +0000 UTC m=+79.174172281" Jul 16 10:33:40.068248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.068201 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xp9mw" podStartSLOduration=18.055686165 podStartE2EDuration="34.068185508s" totalImagesPullingTime="16.012499343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:33:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:40.067841379 +0000 UTC m=+79.204568052" watchObservedRunningTime="2026-07-16 10:33:40.068185508 +0000 UTC m=+79.204912179" Jul 16 10:33:40.093145 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.093106 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2fh28" podStartSLOduration=37.963307106 podStartE2EDuration="43.093090776s" totalImagesPullingTime="5.12978367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:40.091779178 +0000 UTC m=+79.228505851" watchObservedRunningTime="2026-07-16 10:33:40.093090776 +0000 UTC m=+79.229817427" Jul 16 10:33:40.113288 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.113247 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-448ln" podStartSLOduration=42.093598792 podStartE2EDuration="47.113233574s" totalImagesPullingTime="5.019634782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:40.112806755 +0000 UTC m=+79.249533429" watchObservedRunningTime="2026-07-16 10:33:40.113233574 +0000 UTC m=+79.249960246" Jul 16 10:33:40.724735 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.724703 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:40.823103 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:40.823072 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:41.269462 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:41.269406 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:41.370853 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:41.370802 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:41.474870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:41.474841 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:42.382129 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.382101 2568 scope.go:117] "RemoveContainer" containerID="272be47c98c46e8dd3c08448ba3419b0bf7759bf524951371f864464d056151d" Jul 16 10:33:42.396451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.396393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:42.483539 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.483512 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:42.837092 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.837062 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:42.935265 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.935135 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-l74t8"] Jul 16 10:33:42.937897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.937861 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:33:42.941108 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.941089 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-sfxbk\"" Jul 16 10:33:42.941388 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.941369 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 10:33:42.941518 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.941415 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 10:33:42.960567 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.960541 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-l74t8"] Jul 16 10:33:42.977950 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:42.977899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.029919 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.029887 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:33:43.030041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.029998 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ttkps" event={"ID":"3738f020-bb71-4cee-a084-f6a45ec10d00","Type":"ContainerStarted","Data":"306c2544219f9b8da0293e6ad25b2457e726d323206e2f2e8774f048e814c5b4"} Jul 16 10:33:43.030276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.030261 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:33:43.034941 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.034902 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-ttkps" Jul 16 10:33:43.055808 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.055764 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-ttkps" podStartSLOduration=42.513907114 podStartE2EDuration="50.055750039s" totalImagesPullingTime="7.541842925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:43.054494698 +0000 UTC m=+82.191221392" watchObservedRunningTime="2026-07-16 10:33:43.055750039 +0000 UTC m=+82.192476710" Jul 16 10:33:43.064770 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.064742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.082880 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.082853 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wtm9\" (UniqueName: \"kubernetes.io/projected/345f3f95-6ca8-456a-bc44-12c43cef661a-kube-api-access-5wtm9\") pod \"downloads-68c9cbb497-l74t8\" (UID: \"345f3f95-6ca8-456a-bc44-12c43cef661a\") " pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:33:43.184407 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.184329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wtm9\" (UniqueName: \"kubernetes.io/projected/345f3f95-6ca8-456a-bc44-12c43cef661a-kube-api-access-5wtm9\") pod \"downloads-68c9cbb497-l74t8\" (UID: \"345f3f95-6ca8-456a-bc44-12c43cef661a\") " pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:33:43.195161 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.195134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wtm9\" (UniqueName: \"kubernetes.io/projected/345f3f95-6ca8-456a-bc44-12c43cef661a-kube-api-access-5wtm9\") pod \"downloads-68c9cbb497-l74t8\" (UID: \"345f3f95-6ca8-456a-bc44-12c43cef661a\") " pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:33:43.215115 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.215087 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.248051 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.248012 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:33:43.300290 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.300261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.371146 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.371110 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-l74t8"] Jul 16 10:33:43.373220 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:43.373191 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345f3f95_6ca8_456a_bc44_12c43cef661a.slice/crio-1391e6a663aa2b450e3a0e2be30ca01ef36623ddb0cb34668f67bdac7851d953 WatchSource:0}: Error finding container 1391e6a663aa2b450e3a0e2be30ca01ef36623ddb0cb34668f67bdac7851d953: Status 404 returned error can't find the container with id 1391e6a663aa2b450e3a0e2be30ca01ef36623ddb0cb34668f67bdac7851d953 Jul 16 10:33:43.590823 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.590792 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.687537 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.687509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.793372 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.793331 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:43.963687 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:43.963601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:44.036543 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:44.036511 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-l74t8" event={"ID":"345f3f95-6ca8-456a-bc44-12c43cef661a","Type":"ContainerStarted","Data":"1391e6a663aa2b450e3a0e2be30ca01ef36623ddb0cb34668f67bdac7851d953"} Jul 16 10:33:44.239278 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:44.239192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:44.647056 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:44.647020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:44.811869 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:44.811820 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:45.011221 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:45.011147 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:45.114018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:45.113938 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:46.190757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:46.190717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:47.908718 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:47.908690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:48.007246 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.007211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:48.298507 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.298418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:48.424167 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.424131 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc"] Jul 16 10:33:48.429002 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.428977 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-swz8h"] Jul 16 10:33:48.429135 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.429075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.432242 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.432222 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.449564 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.449538 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 10:33:48.459648 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.459623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:48.460714 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.460668 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4pbd2\"" Jul 16 10:33:48.461646 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.461626 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 16 10:33:48.461646 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.461641 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kw2vq\"" Jul 16 10:33:48.461811 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.461646 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 10:33:48.461811 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.461734 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 16 10:33:48.461811 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.461747 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 10:33:48.462352 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.462328 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 16 10:33:48.487229 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.487206 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc"] Jul 16 10:33:48.525259 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525233 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.525386 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525312 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p52g\" (UniqueName: \"kubernetes.io/projected/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-api-access-7p52g\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.525386 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7wq6\" (UniqueName: \"kubernetes.io/projected/013f9ee1-0515-4a70-9e02-e084ce2556b3-kube-api-access-t7wq6\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.525494 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.525494 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525469 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.525588 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.525588 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525580 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.525685 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.525685 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525666 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.525779 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.525697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/013f9ee1-0515-4a70-9e02-e084ce2556b3-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.528361 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.528336 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5zmbw"] Jul 16 10:33:48.531952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.531936 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-swz8h"] Jul 16 10:33:48.532039 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.532011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.540687 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.540614 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 10:33:48.540785 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.540687 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 10:33:48.540840 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.540799 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8gj4m\"" Jul 16 10:33:48.541050 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.541035 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 10:33:48.626841 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.626805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627031 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.626848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.627031 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.626878 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627031 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.626940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627031 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:48.627009 2568 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 16 10:33:48.627255 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:48.627098 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-tls podName:9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:49.127063034 +0000 UTC m=+88.263789695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-swz8h" (UID: "9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89") : secret "kube-state-metrics-tls" not found Jul 16 10:33:48.627255 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627139 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627255 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627255 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627198 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-textfile\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627255 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627238 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627287 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-tls\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627309 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-wtmp\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627408 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/013f9ee1-0515-4a70-9e02-e084ce2556b3-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d096c942-cd20-44f6-9dc8-a358aa13d0f2-metrics-client-ca\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627468 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627519 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627501 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p52g\" (UniqueName: \"kubernetes.io/projected/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-api-access-7p52g\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627533 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-sys\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627557 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fnfx\" (UniqueName: \"kubernetes.io/projected/d096c942-cd20-44f6-9dc8-a358aa13d0f2-kube-api-access-4fnfx\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7wq6\" (UniqueName: \"kubernetes.io/projected/013f9ee1-0515-4a70-9e02-e084ce2556b3-kube-api-access-t7wq6\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.627870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-root\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.627870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627732 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.627870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.628214 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:48.627931 2568 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 16 10:33:48.628214 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.627978 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.628214 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:48.627990 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-tls podName:013f9ee1-0515-4a70-9e02-e084ce2556b3 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:49.127972697 +0000 UTC m=+88.264699350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-lpmpc" (UID: "013f9ee1-0515-4a70-9e02-e084ce2556b3") : secret "openshift-state-metrics-tls" not found Jul 16 10:33:48.628409 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.628389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/013f9ee1-0515-4a70-9e02-e084ce2556b3-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.629622 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.629591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.629947 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.629927 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.660475 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.660453 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7wq6\" (UniqueName: \"kubernetes.io/projected/013f9ee1-0515-4a70-9e02-e084ce2556b3-kube-api-access-t7wq6\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:48.711986 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.711962 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p52g\" (UniqueName: \"kubernetes.io/projected/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-api-access-7p52g\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:48.728154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-root\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728269 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728187 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728269 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-textfile\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728382 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728264 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728382 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-root\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728382 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-tls\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728382 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728335 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-wtmp\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d096c942-cd20-44f6-9dc8-a358aa13d0f2-metrics-client-ca\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-sys\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728448 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fnfx\" (UniqueName: \"kubernetes.io/projected/d096c942-cd20-44f6-9dc8-a358aa13d0f2-kube-api-access-4fnfx\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728519 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-wtmp\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728592 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d096c942-cd20-44f6-9dc8-a358aa13d0f2-sys\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728607 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-textfile\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.728972 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.728943 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.729093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.729066 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d096c942-cd20-44f6-9dc8-a358aa13d0f2-metrics-client-ca\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.730710 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.730688 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-tls\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.730896 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.730875 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d096c942-cd20-44f6-9dc8-a358aa13d0f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.748892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.748865 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:48.793665 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.793635 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fnfx\" (UniqueName: \"kubernetes.io/projected/d096c942-cd20-44f6-9dc8-a358aa13d0f2-kube-api-access-4fnfx\") pod \"node-exporter-5zmbw\" (UID: \"d096c942-cd20-44f6-9dc8-a358aa13d0f2\") " pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.842522 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.842497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5zmbw" Jul 16 10:33:48.857228 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:48.857206 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd096c942_cd20_44f6_9dc8_a358aa13d0f2.slice/crio-e0504b6ea48909f45658da194c9e4103a736114beddad27ac7edd86fbfb1232e WatchSource:0}: Error finding container e0504b6ea48909f45658da194c9e4103a736114beddad27ac7edd86fbfb1232e: Status 404 returned error can't find the container with id e0504b6ea48909f45658da194c9e4103a736114beddad27ac7edd86fbfb1232e Jul 16 10:33:48.944787 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:48.944713 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:49.063587 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.063548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zmbw" event={"ID":"d096c942-cd20-44f6-9dc8-a358aa13d0f2","Type":"ContainerStarted","Data":"e0504b6ea48909f45658da194c9e4103a736114beddad27ac7edd86fbfb1232e"} Jul 16 10:33:49.132468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.132222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:49.132468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.132306 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:49.135380 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.135349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-swz8h\" (UID: \"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:49.135649 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.135616 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/013f9ee1-0515-4a70-9e02-e084ce2556b3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lpmpc\" (UID: \"013f9ee1-0515-4a70-9e02-e084ce2556b3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:49.169330 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.169293 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:49.270283 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.270197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:49.343073 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.343044 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" Jul 16 10:33:49.349178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.349154 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" Jul 16 10:33:49.384351 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.384303 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:49.519839 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.519800 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc"] Jul 16 10:33:49.523172 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:49.523137 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013f9ee1_0515_4a70_9e02_e084ce2556b3.slice/crio-75d9751c76ed4bb440e311ecfab388a5cf0b8c510ad2ca70501a329c5096ed43 WatchSource:0}: Error finding container 75d9751c76ed4bb440e311ecfab388a5cf0b8c510ad2ca70501a329c5096ed43: Status 404 returned error can't find the container with id 75d9751c76ed4bb440e311ecfab388a5cf0b8c510ad2ca70501a329c5096ed43 Jul 16 10:33:49.553736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.553700 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-swz8h"] Jul 16 10:33:49.730475 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.730181 2568 patch_prober.go:28] interesting pod/image-registry-c56d5d8dc-r4tvc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 10:33:49.730475 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.730238 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" podUID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 10:33:49.787330 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:49.785383 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebc82f0_4a7a_4298_95d7_f9c5b20e3c89.slice/crio-7bc3b60ca318ffbaa8e2cd0ff32e4840c2bea60d356e0ddb3a4fbe0b9055a759 WatchSource:0}: Error finding container 7bc3b60ca318ffbaa8e2cd0ff32e4840c2bea60d356e0ddb3a4fbe0b9055a759: Status 404 returned error can't find the container with id 7bc3b60ca318ffbaa8e2cd0ff32e4840c2bea60d356e0ddb3a4fbe0b9055a759 Jul 16 10:33:49.976169 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:49.976095 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:50.023070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.023043 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2fh28" Jul 16 10:33:50.070881 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.070847 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" event={"ID":"013f9ee1-0515-4a70-9e02-e084ce2556b3","Type":"ContainerStarted","Data":"c0b996b76ca2746e6b555281820292f7395b09ed33b70b4e7cee4c5a5225ccc3"} Jul 16 10:33:50.071042 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.070961 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" event={"ID":"013f9ee1-0515-4a70-9e02-e084ce2556b3","Type":"ContainerStarted","Data":"211dec13d4fba0d32a39a907fd26a4501f82d9e584a883788d6296a8b6bb0938"} Jul 16 10:33:50.071042 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.070998 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" event={"ID":"013f9ee1-0515-4a70-9e02-e084ce2556b3","Type":"ContainerStarted","Data":"75d9751c76ed4bb440e311ecfab388a5cf0b8c510ad2ca70501a329c5096ed43"} Jul 16 10:33:50.072833 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.072805 2568 generic.go:356] "Generic (PLEG): container finished" podID="d096c942-cd20-44f6-9dc8-a358aa13d0f2" containerID="bdea51405666383f939dc866cba4cd3848c11d390c21c7746308e06c297841ee" exitCode=0 Jul 16 10:33:50.072976 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.072884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zmbw" event={"ID":"d096c942-cd20-44f6-9dc8-a358aa13d0f2","Type":"ContainerDied","Data":"bdea51405666383f939dc866cba4cd3848c11d390c21c7746308e06c297841ee"} Jul 16 10:33:50.075772 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:50.074663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" event={"ID":"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89","Type":"ContainerStarted","Data":"7bc3b60ca318ffbaa8e2cd0ff32e4840c2bea60d356e0ddb3a4fbe0b9055a759"} Jul 16 10:33:51.082580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.082261 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zmbw" event={"ID":"d096c942-cd20-44f6-9dc8-a358aa13d0f2","Type":"ContainerStarted","Data":"b9665b10404bcad16a09b8c870f78969ad8e16f8df86e212f7d455bea9101fc3"} Jul 16 10:33:51.082580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.082319 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zmbw" event={"ID":"d096c942-cd20-44f6-9dc8-a358aa13d0f2","Type":"ContainerStarted","Data":"fb8812ef34ccb7101e2d44ceb13e8be9103ae0e0ad8f7157e06a47161db6356e"} Jul 16 10:33:51.107469 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.107385 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5zmbw" podStartSLOduration=2.046256345 podStartE2EDuration="3.107364005s" totalImagesPullingTime="982.85775ms" totalInitContainerRuntime="78.24991ms" isStatefulPod=true podCreationTimestamp="2026-07-16 10:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:51.10544196 +0000 UTC m=+90.242168633" watchObservedRunningTime="2026-07-16 10:33:51.107364005 +0000 UTC m=+90.244090671" Jul 16 10:33:51.214986 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.214894 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:51.444726 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.444606 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:51.569540 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.569510 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:51.647496 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.647465 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b897468c7-gkvt7"] Jul 16 10:33:51.652408 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.652390 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.655008 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.654988 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 10:33:51.655289 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.655269 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 10:33:51.655376 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.655320 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 10:33:51.655649 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.655508 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 10:33:51.656162 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.656145 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 10:33:51.657691 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.657575 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qp7fd\"" Jul 16 10:33:51.665782 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.665761 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b897468c7-gkvt7"] Jul 16 10:33:51.755935 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.755592 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7h8\" (UniqueName: \"kubernetes.io/projected/00b801b4-ecf1-4592-bdc6-27717ec91e2f-kube-api-access-7z7h8\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.755935 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.755854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-oauth-serving-cert\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.756932 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.756348 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-oauth-config\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.756932 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.756384 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-service-ca\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.756932 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.756845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-config\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.757564 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.757196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-serving-cert\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.858273 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.858241 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-config\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.858378 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.858300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-serving-cert\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.858378 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.858345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7h8\" (UniqueName: \"kubernetes.io/projected/00b801b4-ecf1-4592-bdc6-27717ec91e2f-kube-api-access-7z7h8\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.858478 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.858388 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-oauth-serving-cert\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.858478 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.858439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-oauth-config\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.858478 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.858464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-service-ca\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.859322 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.859092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-config\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.859322 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.859163 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-oauth-serving-cert\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.859952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.859889 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-service-ca\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.864502 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.864468 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-serving-cert\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.865556 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.865533 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-oauth-config\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.869413 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.869378 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7h8\" (UniqueName: \"kubernetes.io/projected/00b801b4-ecf1-4592-bdc6-27717ec91e2f-kube-api-access-7z7h8\") pod \"console-b897468c7-gkvt7\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:51.964518 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:51.964425 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:33:52.089973 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.089942 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" event={"ID":"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89","Type":"ContainerStarted","Data":"70d101da734c55a37053e68c3f2564095ddbb50c6e0cff4c3e8ee68436b3d16f"} Jul 16 10:33:52.090409 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.090044 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" event={"ID":"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89","Type":"ContainerStarted","Data":"eb6985a677cf328648f51e00435a2ff2dd1caaee439902746a433ffe8cf28278"} Jul 16 10:33:52.090409 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.090079 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" event={"ID":"9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89","Type":"ContainerStarted","Data":"27f55e56d3273b1396af6e0dd3309e354a60df8776dc51a9ed61a6239f9bfbcd"} Jul 16 10:33:52.093649 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.093260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" event={"ID":"013f9ee1-0515-4a70-9e02-e084ce2556b3","Type":"ContainerStarted","Data":"042f8f9ab8c5913e8d63a30e3de74a21e98252dc0fb335097ca7a2f162e3ff89"} Jul 16 10:33:52.110063 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.109998 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-swz8h" podStartSLOduration=2.315664295 podStartE2EDuration="4.109979s" totalImagesPullingTime="1.794314705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:52.109351024 +0000 UTC m=+91.246077695" watchObservedRunningTime="2026-07-16 10:33:52.109979 +0000 UTC m=+91.246705673" Jul 16 10:33:52.115448 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.115428 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b897468c7-gkvt7"] Jul 16 10:33:52.117454 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:33:52.117430 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b801b4_ecf1_4592_bdc6_27717ec91e2f.slice/crio-ec39ff06b00243add69afeca4518d7087adcd62651d59d0f76ae67b33a795166 WatchSource:0}: Error finding container ec39ff06b00243add69afeca4518d7087adcd62651d59d0f76ae67b33a795166: Status 404 returned error can't find the container with id ec39ff06b00243add69afeca4518d7087adcd62651d59d0f76ae67b33a795166 Jul 16 10:33:52.134849 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.134797 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lpmpc" podStartSLOduration=2.481277845 podStartE2EDuration="4.134778616s" totalImagesPullingTime="1.653500771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:52.13250648 +0000 UTC m=+91.269233155" watchObservedRunningTime="2026-07-16 10:33:52.134778616 +0000 UTC m=+91.271505289" Jul 16 10:33:52.955313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.955278 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58"] Jul 16 10:33:52.958673 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.958652 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:52.960529 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.960502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:52.961851 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.961832 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 10:33:52.961966 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.961836 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-ncm2x\"" Jul 16 10:33:52.970522 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.970494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgcfr\" (UniqueName: \"kubernetes.io/projected/72deebca-79f4-4364-a01e-7c29d7bf801b-kube-api-access-zgcfr\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:52.970680 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.970537 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72deebca-79f4-4364-a01e-7c29d7bf801b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:52.970804 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:52.970783 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58"] Jul 16 10:33:53.072090 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.072058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgcfr\" (UniqueName: \"kubernetes.io/projected/72deebca-79f4-4364-a01e-7c29d7bf801b-kube-api-access-zgcfr\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:53.072273 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.072106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72deebca-79f4-4364-a01e-7c29d7bf801b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:53.072344 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:53.072272 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 16 10:33:53.072344 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:33:53.072330 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72deebca-79f4-4364-a01e-7c29d7bf801b-monitoring-plugin-cert podName:72deebca-79f4-4364-a01e-7c29d7bf801b nodeName:}" failed. No retries permitted until 2026-07-16 10:33:53.572308243 +0000 UTC m=+92.709034892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/72deebca-79f4-4364-a01e-7c29d7bf801b-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-tnh58" (UID: "72deebca-79f4-4364-a01e-7c29d7bf801b") : secret "monitoring-plugin-cert" not found Jul 16 10:33:53.076669 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.076594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:53.087353 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.087333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgcfr\" (UniqueName: \"kubernetes.io/projected/72deebca-79f4-4364-a01e-7c29d7bf801b-kube-api-access-zgcfr\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:53.100304 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.100274 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897468c7-gkvt7" event={"ID":"00b801b4-ecf1-4592-bdc6-27717ec91e2f","Type":"ContainerStarted","Data":"ec39ff06b00243add69afeca4518d7087adcd62651d59d0f76ae67b33a795166"} Jul 16 10:33:53.354029 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.353936 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:53.575600 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.575557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72deebca-79f4-4364-a01e-7c29d7bf801b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:53.578451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.578418 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72deebca-79f4-4364-a01e-7c29d7bf801b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tnh58\" (UID: \"72deebca-79f4-4364-a01e-7c29d7bf801b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:53.761755 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.761604 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-84f56f6cff-24rgq"] Jul 16 10:33:53.766539 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.766514 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.771787 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.771761 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 16 10:33:53.771937 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.771896 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 16 10:33:53.772016 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.771960 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 16 10:33:53.772393 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.772370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-prx7b\"" Jul 16 10:33:53.772495 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.772381 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 16 10:33:53.772870 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.772846 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778748 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-secret-telemeter-client\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-serving-certs-ca-bundle\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778853 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wxrs\" (UniqueName: \"kubernetes.io/projected/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-kube-api-access-6wxrs\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-federate-client-tls\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.778993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-telemeter-client-tls\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.779925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.779018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-metrics-client-ca\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.784837 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.784582 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 16 10:33:53.786791 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.786768 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84f56f6cff-24rgq"] Jul 16 10:33:53.870751 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.870722 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:33:53.880023 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.879996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wxrs\" (UniqueName: \"kubernetes.io/projected/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-kube-api-access-6wxrs\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-federate-client-tls\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-telemeter-client-tls\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880124 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-metrics-client-ca\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880312 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-secret-telemeter-client\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880312 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-serving-certs-ca-bundle\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880624 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880599 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.880695 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.880653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.881197 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.881128 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-metrics-client-ca\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.881518 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.881417 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-serving-certs-ca-bundle\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.881950 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.881928 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.883684 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.883300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.883684 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.883421 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-telemeter-client-tls\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.883684 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.883481 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-federate-client-tls\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.883883 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.883863 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-secret-telemeter-client\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:53.891414 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:53.891373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wxrs\" (UniqueName: \"kubernetes.io/projected/61b34bf5-b4fc-4d86-bda4-7c87d0b21a16-kube-api-access-6wxrs\") pod \"telemeter-client-84f56f6cff-24rgq\" (UID: \"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16\") " pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:54.084492 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:54.084451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" Jul 16 10:33:54.509335 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:54.509246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:55.999847 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:55.999817 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:33:56.328227 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:56.328181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:56.422180 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:56.422135 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:57.651541 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:57.651509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:57.752243 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:57.752208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:59.276503 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:59.276466 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:33:59.376269 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:33:59.376235 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:00.847431 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:00.847390 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:00.960134 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:00.960065 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lpgjl" Jul 16 10:34:00.964923 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:00.964883 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:02.489486 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:02.489446 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58"] Jul 16 10:34:02.492700 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:34:02.492669 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72deebca_79f4_4364_a01e_7c29d7bf801b.slice/crio-c86d024e436cd2c01ad787c348479596f093b0b258f7f1b8be40e8a02ad8cc4c WatchSource:0}: Error finding container c86d024e436cd2c01ad787c348479596f093b0b258f7f1b8be40e8a02ad8cc4c: Status 404 returned error can't find the container with id c86d024e436cd2c01ad787c348479596f093b0b258f7f1b8be40e8a02ad8cc4c Jul 16 10:34:02.521018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:02.520988 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84f56f6cff-24rgq"] Jul 16 10:34:02.521726 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:34:02.521687 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b34bf5_b4fc_4d86_bda4_7c87d0b21a16.slice/crio-bc78df8fc11a1c3bd73f32a8860d590000872a47bc98f6361d623ec53d0e806b WatchSource:0}: Error finding container bc78df8fc11a1c3bd73f32a8860d590000872a47bc98f6361d623ec53d0e806b: Status 404 returned error can't find the container with id bc78df8fc11a1c3bd73f32a8860d590000872a47bc98f6361d623ec53d0e806b Jul 16 10:34:02.551990 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:02.551947 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:02.651846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:02.651758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:03.146353 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.146286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-l74t8" event={"ID":"345f3f95-6ca8-456a-bc44-12c43cef661a","Type":"ContainerStarted","Data":"3c7dd22a016824378cfdc17b553cc242758fabe7ea56ca3aea6ef0e4ca9aa923"} Jul 16 10:34:03.146728 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.146703 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:34:03.147852 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.147826 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" event={"ID":"72deebca-79f4-4364-a01e-7c29d7bf801b","Type":"ContainerStarted","Data":"c86d024e436cd2c01ad787c348479596f093b0b258f7f1b8be40e8a02ad8cc4c"} Jul 16 10:34:03.149354 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.149329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" event={"ID":"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16","Type":"ContainerStarted","Data":"bc78df8fc11a1c3bd73f32a8860d590000872a47bc98f6361d623ec53d0e806b"} Jul 16 10:34:03.151037 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.151015 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897468c7-gkvt7" event={"ID":"00b801b4-ecf1-4592-bdc6-27717ec91e2f","Type":"ContainerStarted","Data":"412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43"} Jul 16 10:34:03.164802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.164774 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-l74t8" Jul 16 10:34:03.180650 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.180591 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-l74t8" podStartSLOduration=2.201373763 podStartE2EDuration="21.180569789s" totalImagesPullingTime="18.979196026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:03.180050994 +0000 UTC m=+102.316777668" watchObservedRunningTime="2026-07-16 10:34:03.180569789 +0000 UTC m=+102.317296462" Jul 16 10:34:03.187234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.187194 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c56d5d8dc-r4tvc"] Jul 16 10:34:03.248010 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.247955 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b897468c7-gkvt7" podStartSLOduration=2.016134668 podStartE2EDuration="12.24793587s" totalImagesPullingTime="10.231801202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:03.239663815 +0000 UTC m=+102.376390497" watchObservedRunningTime="2026-07-16 10:34:03.24793587 +0000 UTC m=+102.384662543" Jul 16 10:34:03.350284 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.350198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:03.441858 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.441739 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:03.520487 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.520426 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:03.864652 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.864578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:03.966085 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:03.965965 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:04.045190 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:04.045155 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:05.304862 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:05.304825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:05.422881 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:05.422845 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:05.987627 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:05.987549 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:06.139945 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:06.139879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:06.169412 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:06.169373 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" event={"ID":"72deebca-79f4-4364-a01e-7c29d7bf801b","Type":"ContainerStarted","Data":"709dbf71622b1afcd3912910c40cc10748cc56d291524ce7375c9cb03bf834cc"} Jul 16 10:34:06.169677 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:06.169616 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:34:06.171151 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:06.171094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" event={"ID":"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16","Type":"ContainerStarted","Data":"b7d3b6589f9ed0040dc4efe767deb40a4be7f1767b5be43d284db927bd4a1064"} Jul 16 10:34:06.175638 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:06.175611 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" Jul 16 10:34:06.196950 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:06.196896 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tnh58" podStartSLOduration=11.245657712 podStartE2EDuration="14.196881995s" totalImagesPullingTime="2.951224283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:06.195817337 +0000 UTC m=+105.332544011" watchObservedRunningTime="2026-07-16 10:34:06.196881995 +0000 UTC m=+105.333608667" Jul 16 10:34:08.183590 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:08.183551 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" event={"ID":"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16","Type":"ContainerStarted","Data":"9a4d24725a9c86f3686309561405fc29b3d3a21eb4e743b1be45fa37051dccf5"} Jul 16 10:34:08.184056 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:08.183652 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" event={"ID":"61b34bf5-b4fc-4d86-bda4-7c87d0b21a16","Type":"ContainerStarted","Data":"0f9178773289c5573b7c206057acbe960ba79b6339725c4c4b00c952dd5e15bd"} Jul 16 10:34:08.240234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:08.240182 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-84f56f6cff-24rgq" podStartSLOduration=10.430959792 podStartE2EDuration="15.240164102s" totalImagesPullingTime="4.80920431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:08.238851956 +0000 UTC m=+107.375578632" watchObservedRunningTime="2026-07-16 10:34:08.240164102 +0000 UTC m=+107.376890773" Jul 16 10:34:08.433144 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:08.433102 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:09.483557 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:09.483519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:09.547358 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:09.547327 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b897468c7-gkvt7"] Jul 16 10:34:09.614665 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:09.614633 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:09.899424 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:09.899391 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:10.018177 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:10.018144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:11.283606 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:11.283558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:11.408973 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:11.408941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:11.965134 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:11.965100 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:34:12.064189 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:12.064147 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:12.227287 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:12.227205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:13.113322 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:13.113281 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:13.213952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:13.213922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:15.491867 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:15.491837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:15.597869 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:15.597841 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:17.221781 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:17.221750 2568 generic.go:356] "Generic (PLEG): container finished" podID="707e6bfc-4cad-4b9e-9712-93f5e1970cf1" containerID="cda1915651ed4e638a2fd1640b96d286938a83d1c9211a534754c2a4a0113fca" exitCode=0 Jul 16 10:34:17.222171 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:17.221823 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" event={"ID":"707e6bfc-4cad-4b9e-9712-93f5e1970cf1","Type":"ContainerDied","Data":"cda1915651ed4e638a2fd1640b96d286938a83d1c9211a534754c2a4a0113fca"} Jul 16 10:34:17.222171 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:17.222145 2568 scope.go:117] "RemoveContainer" containerID="cda1915651ed4e638a2fd1640b96d286938a83d1c9211a534754c2a4a0113fca" Jul 16 10:34:18.230531 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:18.230497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-vs4xw" event={"ID":"707e6bfc-4cad-4b9e-9712-93f5e1970cf1","Type":"ContainerStarted","Data":"b3972a07c0128eb5e4dda43c9fbc0f9e264b777c1c689bcce46489615349020a"} Jul 16 10:34:20.490398 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:20.490367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:20.590445 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:20.590417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:22.247873 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:22.247838 2568 generic.go:356] "Generic (PLEG): container finished" podID="e11cee50-17ab-4283-a72b-50a08cd6946b" containerID="a72e145cb921b14a7c5500beb28bbaf642245dc936c8aab4f798d3b10a761cd4" exitCode=0 Jul 16 10:34:22.247873 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:22.247876 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" event={"ID":"e11cee50-17ab-4283-a72b-50a08cd6946b","Type":"ContainerDied","Data":"a72e145cb921b14a7c5500beb28bbaf642245dc936c8aab4f798d3b10a761cd4"} Jul 16 10:34:22.248306 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:22.248213 2568 scope.go:117] "RemoveContainer" containerID="a72e145cb921b14a7c5500beb28bbaf642245dc936c8aab4f798d3b10a761cd4" Jul 16 10:34:23.253705 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:23.253672 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v8v66" event={"ID":"e11cee50-17ab-4283-a72b-50a08cd6946b","Type":"ContainerStarted","Data":"175719ac5a834e07ac7b5a717bcd1fd16265bbe92d6ba087fc7c5b610bc6263f"} Jul 16 10:34:25.731197 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:25.731156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:25.831336 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:25.831304 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:34:28.217276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.217233 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" podUID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" containerName="registry" containerID="cri-o://371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1" gracePeriod=30 Jul 16 10:34:28.506490 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.506460 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:34:28.588275 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588246 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588299 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-image-registry-private-configuration\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588335 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dwvx\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-kube-api-access-4dwvx\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588364 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-certificates\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588391 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-ca-trust-extracted\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588426 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-trusted-ca\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588693 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588482 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-installation-pull-secrets\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.588693 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.588534 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-bound-sa-token\") pod \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\" (UID: \"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5\") " Jul 16 10:34:28.589732 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.589392 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:28.589732 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.589644 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:28.590965 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.590932 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:34:28.591095 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.591026 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:34:28.591546 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.591518 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:34:28.591647 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.591593 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:34:28.591706 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.591655 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-kube-api-access-4dwvx" (OuterVolumeSpecName: "kube-api-access-4dwvx") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "kube-api-access-4dwvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:34:28.598820 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.598796 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" (UID: "2f3e0792-5e71-481e-9b6b-db9c8f80d8c5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:34:28.690127 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690094 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-installation-pull-secrets\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690127 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690123 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-bound-sa-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690127 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690133 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-tls\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690412 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690144 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-image-registry-private-configuration\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690412 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690153 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dwvx\" (UniqueName: \"kubernetes.io/projected/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-kube-api-access-4dwvx\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690412 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690162 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-registry-certificates\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690412 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690172 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-ca-trust-extracted\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:28.690412 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:28.690180 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5-trusted-ca\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:29.283238 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.283196 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" containerID="371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1" exitCode=0 Jul 16 10:34:29.283715 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.283284 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" event={"ID":"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5","Type":"ContainerDied","Data":"371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1"} Jul 16 10:34:29.283715 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.283335 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" event={"ID":"2f3e0792-5e71-481e-9b6b-db9c8f80d8c5","Type":"ContainerDied","Data":"c8c568b1e24504e4ae1d3cbf29575a36dec084fba2fd9532ac94422ff7208006"} Jul 16 10:34:29.283715 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.283366 2568 scope.go:117] "RemoveContainer" containerID="371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1" Jul 16 10:34:29.283715 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.283537 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c56d5d8dc-r4tvc" Jul 16 10:34:29.296622 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.296590 2568 scope.go:117] "RemoveContainer" containerID="371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1" Jul 16 10:34:29.297301 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:34:29.296891 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1\": container with ID starting with 371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1 not found: ID does not exist" containerID="371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1" Jul 16 10:34:29.297381 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.297318 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1"} err="failed to get container status \"371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1\": rpc error: code = NotFound desc = could not find container \"371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1\": container with ID starting with 371b1c8b75f664a50cd1ada6ecdf30b8e5dd9e01b0bd8ebaa2f36e60a49ccbc1 not found: ID does not exist" Jul 16 10:34:29.308320 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.308296 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c56d5d8dc-r4tvc"] Jul 16 10:34:29.319314 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.319291 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c56d5d8dc-r4tvc"] Jul 16 10:34:29.390172 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:29.390133 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" path="/var/lib/kubelet/pods/2f3e0792-5e71-481e-9b6b-db9c8f80d8c5/volumes" Jul 16 10:34:34.572031 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:34.571988 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b897468c7-gkvt7" podUID="00b801b4-ecf1-4592-bdc6-27717ec91e2f" containerName="console" containerID="cri-o://412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43" gracePeriod=15 Jul 16 10:34:34.861835 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:34.861811 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:34:35.041577 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.041543 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z7h8\" (UniqueName: \"kubernetes.io/projected/00b801b4-ecf1-4592-bdc6-27717ec91e2f-kube-api-access-7z7h8\") pod \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " Jul 16 10:34:35.041733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.041589 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-oauth-serving-cert\") pod \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " Jul 16 10:34:35.041733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.041629 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-service-ca\") pod \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " Jul 16 10:34:35.041733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.041666 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-serving-cert\") pod \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " Jul 16 10:34:35.041733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.041696 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-config\") pod \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " Jul 16 10:34:35.041733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.041716 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-oauth-config\") pod \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\" (UID: \"00b801b4-ecf1-4592-bdc6-27717ec91e2f\") " Jul 16 10:34:35.042099 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.042064 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-service-ca" (OuterVolumeSpecName: "service-ca") pod "00b801b4-ecf1-4592-bdc6-27717ec91e2f" (UID: "00b801b4-ecf1-4592-bdc6-27717ec91e2f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:35.042206 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.042069 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-config" (OuterVolumeSpecName: "console-config") pod "00b801b4-ecf1-4592-bdc6-27717ec91e2f" (UID: "00b801b4-ecf1-4592-bdc6-27717ec91e2f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:35.042206 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.042063 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "00b801b4-ecf1-4592-bdc6-27717ec91e2f" (UID: "00b801b4-ecf1-4592-bdc6-27717ec91e2f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:35.043756 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.043727 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "00b801b4-ecf1-4592-bdc6-27717ec91e2f" (UID: "00b801b4-ecf1-4592-bdc6-27717ec91e2f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:34:35.043846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.043789 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b801b4-ecf1-4592-bdc6-27717ec91e2f-kube-api-access-7z7h8" (OuterVolumeSpecName: "kube-api-access-7z7h8") pod "00b801b4-ecf1-4592-bdc6-27717ec91e2f" (UID: "00b801b4-ecf1-4592-bdc6-27717ec91e2f"). InnerVolumeSpecName "kube-api-access-7z7h8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:34:35.043846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.043789 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "00b801b4-ecf1-4592-bdc6-27717ec91e2f" (UID: "00b801b4-ecf1-4592-bdc6-27717ec91e2f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:34:35.142234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.142177 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-serving-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:35.142234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.142200 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-config\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:35.142234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.142209 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00b801b4-ecf1-4592-bdc6-27717ec91e2f-console-oauth-config\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:35.142234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.142217 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z7h8\" (UniqueName: \"kubernetes.io/projected/00b801b4-ecf1-4592-bdc6-27717ec91e2f-kube-api-access-7z7h8\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:35.142234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.142226 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-oauth-serving-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:35.142234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.142234 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00b801b4-ecf1-4592-bdc6-27717ec91e2f-service-ca\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:34:35.308556 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.308522 2568 generic.go:356] "Generic (PLEG): container finished" podID="00b801b4-ecf1-4592-bdc6-27717ec91e2f" containerID="412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43" exitCode=0 Jul 16 10:34:35.308705 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.308567 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897468c7-gkvt7" Jul 16 10:34:35.308705 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.308582 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897468c7-gkvt7" event={"ID":"00b801b4-ecf1-4592-bdc6-27717ec91e2f","Type":"ContainerDied","Data":"412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43"} Jul 16 10:34:35.308705 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.308631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897468c7-gkvt7" event={"ID":"00b801b4-ecf1-4592-bdc6-27717ec91e2f","Type":"ContainerDied","Data":"ec39ff06b00243add69afeca4518d7087adcd62651d59d0f76ae67b33a795166"} Jul 16 10:34:35.308705 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.308658 2568 scope.go:117] "RemoveContainer" containerID="412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43" Jul 16 10:34:35.317150 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.317128 2568 scope.go:117] "RemoveContainer" containerID="412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43" Jul 16 10:34:35.317402 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:34:35.317380 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43\": container with ID starting with 412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43 not found: ID does not exist" containerID="412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43" Jul 16 10:34:35.317468 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.317411 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43"} err="failed to get container status \"412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43\": rpc error: code = NotFound desc = could not find container \"412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43\": container with ID starting with 412e795434f00435ee98d7eaedfd0a9946da6b053b5b433ec62eb4bbd6978c43 not found: ID does not exist" Jul 16 10:34:35.330757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.330737 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b897468c7-gkvt7"] Jul 16 10:34:35.336396 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.336366 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b897468c7-gkvt7"] Jul 16 10:34:35.386289 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:35.386264 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b801b4-ecf1-4592-bdc6-27717ec91e2f" path="/var/lib/kubelet/pods/00b801b4-ecf1-4592-bdc6-27717ec91e2f/volumes" Jul 16 10:34:37.318344 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:37.318312 2568 generic.go:356] "Generic (PLEG): container finished" podID="34af818c-77a4-4346-8165-a15195c93008" containerID="6342115fe6ba5027614779723df5b273590f0f6620e45e98a6f05af061a20d3c" exitCode=0 Jul 16 10:34:37.318804 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:37.318379 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" event={"ID":"34af818c-77a4-4346-8165-a15195c93008","Type":"ContainerDied","Data":"6342115fe6ba5027614779723df5b273590f0f6620e45e98a6f05af061a20d3c"} Jul 16 10:34:37.318883 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:37.318806 2568 scope.go:117] "RemoveContainer" containerID="6342115fe6ba5027614779723df5b273590f0f6620e45e98a6f05af061a20d3c" Jul 16 10:34:38.324267 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:34:38.324233 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-cqjb9" event={"ID":"34af818c-77a4-4346-8165-a15195c93008","Type":"ContainerStarted","Data":"da45ed0e73662db59722384d4f96840a80aac72af0c4dadc83741370fb436a77"} Jul 16 10:35:04.431364 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:04.431329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:04.507851 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:04.507813 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:14.499656 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:14.499567 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:16.440493 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:16.440447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:16.797372 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:16.797280 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:25.714156 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:25.714126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:25.816837 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:25.816796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:25.895848 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:25.895823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:43.046527 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046498 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b7b79598-zj6v5"] Jul 16 10:35:43.047117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046788 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b801b4-ecf1-4592-bdc6-27717ec91e2f" containerName="console" Jul 16 10:35:43.047117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046798 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b801b4-ecf1-4592-bdc6-27717ec91e2f" containerName="console" Jul 16 10:35:43.047117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046807 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" containerName="registry" Jul 16 10:35:43.047117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046812 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" containerName="registry" Jul 16 10:35:43.047117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046870 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f3e0792-5e71-481e-9b6b-db9c8f80d8c5" containerName="registry" Jul 16 10:35:43.047117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.046882 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b801b4-ecf1-4592-bdc6-27717ec91e2f" containerName="console" Jul 16 10:35:43.048739 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.048716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.051579 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.051558 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 10:35:43.051814 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.051795 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 10:35:43.052852 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.052833 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qp7fd\"" Jul 16 10:35:43.052992 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.052844 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 10:35:43.053067 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.053031 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 10:35:43.053067 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.052867 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 10:35:43.057085 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.057066 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 10:35:43.069697 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.069676 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b7b79598-zj6v5"] Jul 16 10:35:43.117504 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.117477 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:43.148278 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-serving-cert\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.148384 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-console-config\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.148384 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-oauth-serving-cert\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.148384 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148318 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ws2r\" (UniqueName: \"kubernetes.io/projected/36e53de7-356e-496c-811c-3515613b0c9b-kube-api-access-8ws2r\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.148540 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-service-ca\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.148540 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148493 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-oauth-config\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.148605 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.148555 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-trusted-ca-bundle\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.204637 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.204608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:43.249241 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249208 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-oauth-config\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.249383 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-trusted-ca-bundle\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.249383 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249315 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-serving-cert\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.249383 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-console-config\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.249383 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-oauth-serving-cert\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.249583 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249392 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ws2r\" (UniqueName: \"kubernetes.io/projected/36e53de7-356e-496c-811c-3515613b0c9b-kube-api-access-8ws2r\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.249583 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.249427 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-service-ca\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.250193 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.250138 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-console-config\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.250302 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.250208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-oauth-serving-cert\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.250302 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.250238 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-service-ca\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.250574 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.250555 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-trusted-ca-bundle\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.251819 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.251798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-oauth-config\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.252054 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.252036 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-serving-cert\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.259456 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.259437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ws2r\" (UniqueName: \"kubernetes.io/projected/36e53de7-356e-496c-811c-3515613b0c9b-kube-api-access-8ws2r\") pod \"console-77b7b79598-zj6v5\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.358259 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.358223 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:43.488458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.488428 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b7b79598-zj6v5"] Jul 16 10:35:43.491352 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:35:43.491324 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e53de7_356e_496c_811c_3515613b0c9b.slice/crio-be81d92cd95c2569818ccee6f59641ae79e422a3c4f5909143161a69302534f0 WatchSource:0}: Error finding container be81d92cd95c2569818ccee6f59641ae79e422a3c4f5909143161a69302534f0: Status 404 returned error can't find the container with id be81d92cd95c2569818ccee6f59641ae79e422a3c4f5909143161a69302534f0 Jul 16 10:35:43.570288 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.570263 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7b79598-zj6v5" event={"ID":"36e53de7-356e-496c-811c-3515613b0c9b","Type":"ContainerStarted","Data":"89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78"} Jul 16 10:35:43.570408 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.570326 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7b79598-zj6v5" event={"ID":"36e53de7-356e-496c-811c-3515613b0c9b","Type":"ContainerStarted","Data":"be81d92cd95c2569818ccee6f59641ae79e422a3c4f5909143161a69302534f0"} Jul 16 10:35:43.592754 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:43.592694 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b7b79598-zj6v5" podStartSLOduration=0.592678413 podStartE2EDuration="592.678413ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:35:43.591403614 +0000 UTC m=+202.728130285" watchObservedRunningTime="2026-07-16 10:35:43.592678413 +0000 UTC m=+202.729405086" Jul 16 10:35:47.905667 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:47.905633 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:47.977860 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:47.977833 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:53.358521 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.358487 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:53.358997 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.358543 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:53.363360 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.363340 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:53.611362 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.611283 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:35:53.768305 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.768273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:53.870065 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.869986 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:35:53.951927 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:35:53.951874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:19.618995 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:19.618959 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:19.704266 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:19.704231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:39.173091 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:39.173056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:39.236597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:39.236564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:41.442567 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:41.442534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:42.276149 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:42.276110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:44.705636 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:44.705554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:45.247998 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:45.247964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:45.349567 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:45.349541 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:46.340354 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:46.340323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:46.451683 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:46.451654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:47.267501 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:47.267469 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:47.428110 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:47.428080 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:49.587432 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:49.587403 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:49.807211 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:49.807178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:50.074439 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.074408 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cf44z"] Jul 16 10:36:50.076445 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.076429 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.079191 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.079172 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 10:36:50.079338 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.079302 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 10:36:50.079391 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.079377 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bfpfr\"" Jul 16 10:36:50.089006 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.088979 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cf44z"] Jul 16 10:36:50.173473 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.173444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvnjz\" (UniqueName: \"kubernetes.io/projected/f8429933-ef1b-4b42-98df-7909c451bc5b-kube-api-access-jvnjz\") pod \"cert-manager-cainjector-559cdd8c87-cf44z\" (UID: \"f8429933-ef1b-4b42-98df-7909c451bc5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.173597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.173501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8429933-ef1b-4b42-98df-7909c451bc5b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cf44z\" (UID: \"f8429933-ef1b-4b42-98df-7909c451bc5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.274800 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.274769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvnjz\" (UniqueName: \"kubernetes.io/projected/f8429933-ef1b-4b42-98df-7909c451bc5b-kube-api-access-jvnjz\") pod \"cert-manager-cainjector-559cdd8c87-cf44z\" (UID: \"f8429933-ef1b-4b42-98df-7909c451bc5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.274982 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.274948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8429933-ef1b-4b42-98df-7909c451bc5b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cf44z\" (UID: \"f8429933-ef1b-4b42-98df-7909c451bc5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.283653 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.283631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8429933-ef1b-4b42-98df-7909c451bc5b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cf44z\" (UID: \"f8429933-ef1b-4b42-98df-7909c451bc5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.283789 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.283768 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvnjz\" (UniqueName: \"kubernetes.io/projected/f8429933-ef1b-4b42-98df-7909c451bc5b-kube-api-access-jvnjz\") pod \"cert-manager-cainjector-559cdd8c87-cf44z\" (UID: \"f8429933-ef1b-4b42-98df-7909c451bc5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.394054 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.393988 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" Jul 16 10:36:50.518539 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.518514 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cf44z"] Jul 16 10:36:50.520443 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:36:50.520415 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8429933_ef1b_4b42_98df_7909c451bc5b.slice/crio-babc063708234c018053b37cf99d2feafdb655abf8f56175104998858bbc8ed5 WatchSource:0}: Error finding container babc063708234c018053b37cf99d2feafdb655abf8f56175104998858bbc8ed5: Status 404 returned error can't find the container with id babc063708234c018053b37cf99d2feafdb655abf8f56175104998858bbc8ed5 Jul 16 10:36:50.827148 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:50.827117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" event={"ID":"f8429933-ef1b-4b42-98df-7909c451bc5b","Type":"ContainerStarted","Data":"babc063708234c018053b37cf99d2feafdb655abf8f56175104998858bbc8ed5"} Jul 16 10:36:52.241247 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:52.241209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:52.314147 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:52.314111 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:53.842320 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:53.842283 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" event={"ID":"f8429933-ef1b-4b42-98df-7909c451bc5b","Type":"ContainerStarted","Data":"5b7ffd8d6d61093f7929a05ad68d4aeaa07f95dc22f274da840af2b1cc5c3860"} Jul 16 10:36:53.866990 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:53.866932 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cf44z" podStartSLOduration=0.766241855 podStartE2EDuration="3.866885997s" totalImagesPullingTime="3.100644142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:36:53.866316985 +0000 UTC m=+273.003043659" watchObservedRunningTime="2026-07-16 10:36:53.866885997 +0000 UTC m=+273.003612670" Jul 16 10:36:57.208300 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.208258 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:57.510093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.510020 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fpqhj"] Jul 16 10:36:57.512073 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.512053 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.518616 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.518595 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xltxk\"" Jul 16 10:36:57.525283 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.525259 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fpqhj"] Jul 16 10:36:57.531834 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.531813 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lzcx\" (UniqueName: \"kubernetes.io/projected/e2a55748-1faa-48cf-b085-55ee69ef14a7-kube-api-access-5lzcx\") pod \"cert-manager-55b4d954d5-fpqhj\" (UID: \"e2a55748-1faa-48cf-b085-55ee69ef14a7\") " pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.531974 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.531876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2a55748-1faa-48cf-b085-55ee69ef14a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-fpqhj\" (UID: \"e2a55748-1faa-48cf-b085-55ee69ef14a7\") " pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.632774 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.632752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2a55748-1faa-48cf-b085-55ee69ef14a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-fpqhj\" (UID: \"e2a55748-1faa-48cf-b085-55ee69ef14a7\") " pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.632884 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.632800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lzcx\" (UniqueName: \"kubernetes.io/projected/e2a55748-1faa-48cf-b085-55ee69ef14a7-kube-api-access-5lzcx\") pod \"cert-manager-55b4d954d5-fpqhj\" (UID: \"e2a55748-1faa-48cf-b085-55ee69ef14a7\") " pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.643891 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.643865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lzcx\" (UniqueName: \"kubernetes.io/projected/e2a55748-1faa-48cf-b085-55ee69ef14a7-kube-api-access-5lzcx\") pod \"cert-manager-55b4d954d5-fpqhj\" (UID: \"e2a55748-1faa-48cf-b085-55ee69ef14a7\") " pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.643891 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.643883 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2a55748-1faa-48cf-b085-55ee69ef14a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-fpqhj\" (UID: \"e2a55748-1faa-48cf-b085-55ee69ef14a7\") " pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.821087 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.821060 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fpqhj" Jul 16 10:36:57.945670 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:57.945615 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fpqhj"] Jul 16 10:36:57.948177 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:36:57.948148 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2a55748_1faa_48cf_b085_55ee69ef14a7.slice/crio-37eb2c6812cd20d7677ca52308117402899f1d8be6775b390426f469bfbdfd9f WatchSource:0}: Error finding container 37eb2c6812cd20d7677ca52308117402899f1d8be6775b390426f469bfbdfd9f: Status 404 returned error can't find the container with id 37eb2c6812cd20d7677ca52308117402899f1d8be6775b390426f469bfbdfd9f Jul 16 10:36:58.864992 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:58.864958 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fpqhj" event={"ID":"e2a55748-1faa-48cf-b085-55ee69ef14a7","Type":"ContainerStarted","Data":"b988efc56fb8a556c6137a22687b31f4c172512845a67405cf2e722987c5937d"} Jul 16 10:36:58.864992 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:58.864996 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fpqhj" event={"ID":"e2a55748-1faa-48cf-b085-55ee69ef14a7","Type":"ContainerStarted","Data":"37eb2c6812cd20d7677ca52308117402899f1d8be6775b390426f469bfbdfd9f"} Jul 16 10:36:58.885409 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:58.885353 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fpqhj" podStartSLOduration=1.885339896 podStartE2EDuration="1.885339896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:36:58.884747896 +0000 UTC m=+278.021474570" watchObservedRunningTime="2026-07-16 10:36:58.885339896 +0000 UTC m=+278.022066568" Jul 16 10:36:59.358709 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:59.358661 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:36:59.457022 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:36:59.456988 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:04.755988 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:04.755947 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:04.856060 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:04.856028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:06.820428 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:06.820392 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:09.439005 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:09.438976 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:09.519282 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:09.519239 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:21.214149 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:21.214119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:37:21.214780 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:21.214119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:37:21.217470 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:21.217450 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:37:21.217470 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:21.217465 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:37:21.227690 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:21.227671 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 10:37:22.146377 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:22.146341 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:23.269863 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.269834 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2"] Jul 16 10:37:23.272149 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.272133 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.275474 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.275451 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 10:37:23.275620 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.275599 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 10:37:23.275965 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.275602 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 10:37:23.275965 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.275924 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 10:37:23.276191 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.276176 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j9zwf\"" Jul 16 10:37:23.307101 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.307075 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2"] Jul 16 10:37:23.337323 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.337298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.337445 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.337329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.337445 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.337354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.337557 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.337468 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whh5w\" (UniqueName: \"kubernetes.io/projected/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-kube-api-access-whh5w\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.337557 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.337532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.370735 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.370710 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz"] Jul 16 10:37:23.372839 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.372824 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.378066 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.378044 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 10:37:23.378066 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.378048 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 10:37:23.378240 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.378140 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 10:37:23.378240 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.378140 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 10:37:23.378343 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.378270 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:37:23.378343 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.378300 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bzbdm\"" Jul 16 10:37:23.404350 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.404327 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz"] Jul 16 10:37:23.438091 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.438212 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.438212 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.438212 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438159 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-metrics-cert\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.438212 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438192 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whh5w\" (UniqueName: \"kubernetes.io/projected/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-kube-api-access-whh5w\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.438367 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-manager-config\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.438447 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.438520 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgs74\" (UniqueName: \"kubernetes.io/projected/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-kube-api-access-wgs74\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.438520 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438503 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-cert\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.438645 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.438531 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.440456 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.440432 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.440578 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.440523 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.440578 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.440553 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.453033 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.453008 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whh5w\" (UniqueName: \"kubernetes.io/projected/574cbc13-8e5e-47d1-a0ab-c4ca563a5a95-kube-api-access-whh5w\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-cmdh2\" (UID: \"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.539119 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.539036 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-manager-config\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.539119 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.539081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgs74\" (UniqueName: \"kubernetes.io/projected/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-kube-api-access-wgs74\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.539119 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.539101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-cert\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.539470 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.539159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-metrics-cert\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.539650 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.539628 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-manager-config\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.541453 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.541435 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-cert\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.541499 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.541477 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-metrics-cert\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.549289 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.549269 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgs74\" (UniqueName: \"kubernetes.io/projected/ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c-kube-api-access-wgs74\") pod \"lws-controller-manager-85564789c6-7xlpz\" (UID: \"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.582187 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.582154 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:23.682401 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.682377 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:23.740683 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.740656 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2"] Jul 16 10:37:23.742630 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:37:23.742585 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574cbc13_8e5e_47d1_a0ab_c4ca563a5a95.slice/crio-57363f49143829eb38c65931fe3b223032a5a9245d2f455a783c887927aa38b1 WatchSource:0}: Error finding container 57363f49143829eb38c65931fe3b223032a5a9245d2f455a783c887927aa38b1: Status 404 returned error can't find the container with id 57363f49143829eb38c65931fe3b223032a5a9245d2f455a783c887927aa38b1 Jul 16 10:37:23.745329 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.745310 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 10:37:23.826554 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.826388 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz"] Jul 16 10:37:23.828942 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:37:23.828901 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6fbf6b_b6e2_4aab_9da1_1dd48485aa3c.slice/crio-c9cf3d8bcea6020f4ea4f03890f4de8649801a0623d82b422222e451f0a1ba8f WatchSource:0}: Error finding container c9cf3d8bcea6020f4ea4f03890f4de8649801a0623d82b422222e451f0a1ba8f: Status 404 returned error can't find the container with id c9cf3d8bcea6020f4ea4f03890f4de8649801a0623d82b422222e451f0a1ba8f Jul 16 10:37:23.959258 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.959223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:23.965832 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.965804 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" event={"ID":"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95","Type":"ContainerStarted","Data":"57363f49143829eb38c65931fe3b223032a5a9245d2f455a783c887927aa38b1"} Jul 16 10:37:23.966951 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:23.966931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" event={"ID":"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c","Type":"ContainerStarted","Data":"c9cf3d8bcea6020f4ea4f03890f4de8649801a0623d82b422222e451f0a1ba8f"} Jul 16 10:37:24.067200 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:24.067170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:24.190420 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:24.190340 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:27.994036 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:27.994003 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" event={"ID":"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95","Type":"ContainerStarted","Data":"9635e0c08eff94dcc002da5409b1f7aaff34ad67f1a0f7c047c007a8e3b055e8"} Jul 16 10:37:27.995557 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:27.995528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" event={"ID":"ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c","Type":"ContainerStarted","Data":"69af76b8ca056cec3732e35a4db4085e483c255f90ff2228a834c5469a79f3fe"} Jul 16 10:37:27.995687 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:27.995576 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:28.051489 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:28.051437 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" podStartSLOduration=0.995225788 podStartE2EDuration="5.051421855s" totalImagesPullingTime="4.056196067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:28.050526125 +0000 UTC m=+307.187252798" watchObservedRunningTime="2026-07-16 10:37:28.051421855 +0000 UTC m=+307.188148563" Jul 16 10:37:30.006082 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:30.006052 2568 generic.go:356] "Generic (PLEG): container finished" podID="574cbc13-8e5e-47d1-a0ab-c4ca563a5a95" containerID="9635e0c08eff94dcc002da5409b1f7aaff34ad67f1a0f7c047c007a8e3b055e8" exitCode=0 Jul 16 10:37:30.006449 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:30.006131 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" event={"ID":"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95","Type":"ContainerDied","Data":"9635e0c08eff94dcc002da5409b1f7aaff34ad67f1a0f7c047c007a8e3b055e8"} Jul 16 10:37:31.011901 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:31.011868 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" event={"ID":"574cbc13-8e5e-47d1-a0ab-c4ca563a5a95","Type":"ContainerStarted","Data":"7422cda71e2f9a002de9c911fbf9fb6a564ec46bf55cdc81430aef755d2a08f3"} Jul 16 10:37:31.012302 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:31.012091 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:31.036733 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:31.036688 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" podStartSLOduration=2.525585839 podStartE2EDuration="8.036672352s" totalImagesPullingTime="4.418354015s" totalInitContainerRuntime="1.092732498s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:31.035281953 +0000 UTC m=+310.172008651" watchObservedRunningTime="2026-07-16 10:37:31.036672352 +0000 UTC m=+310.173399024" Jul 16 10:37:39.002964 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:39.002934 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-85564789c6-7xlpz" Jul 16 10:37:42.018387 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:42.018358 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-cmdh2" Jul 16 10:37:44.203032 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:44.202998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:44.351850 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:44.351813 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:45.444290 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:45.444258 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:45.729000 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:45.728877 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:45.834508 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:45.834480 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:45.947772 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:45.947736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:46.087372 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:46.087342 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:46.604448 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:46.604416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:47.204392 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:47.204347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:47.577600 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:47.577571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:47.725063 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:47.725033 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:49.662218 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:49.662189 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:50.844482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:50.844449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:53.829619 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:53.829590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:54.020188 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:54.020143 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:55.639213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.639169 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf"] Jul 16 10:37:55.642788 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.642766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.649673 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.649655 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 10:37:55.650058 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.650043 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cgcnw\"" Jul 16 10:37:55.664865 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.664842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:55.758766 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.758738 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf"] Jul 16 10:37:55.810442 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810481 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810498 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810561 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcp4p\" (UniqueName: \"kubernetes.io/projected/f3ff2214-1499-405a-af0d-f397817cc5d9-kube-api-access-tcp4p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810566 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.810720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.810672 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3ff2214-1499-405a-af0d-f397817cc5d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912047 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.911980 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912047 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912047 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912041 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912056 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcp4p\" (UniqueName: \"kubernetes.io/projected/f3ff2214-1499-405a-af0d-f397817cc5d9-kube-api-access-tcp4p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912248 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3ff2214-1499-405a-af0d-f397817cc5d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912535 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912738 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912714 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912802 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912745 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912844 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912827 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.912936 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.912921 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3ff2214-1499-405a-af0d-f397817cc5d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.914318 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.914296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.914455 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.914436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.934437 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.934411 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3ff2214-1499-405a-af0d-f397817cc5d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.939850 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.939826 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcp4p\" (UniqueName: \"kubernetes.io/projected/f3ff2214-1499-405a-af0d-f397817cc5d9-kube-api-access-tcp4p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mztsf\" (UID: \"f3ff2214-1499-405a-af0d-f397817cc5d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:55.952721 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:55.952699 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:56.212129 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:56.212104 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf"] Jul 16 10:37:56.214675 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:37:56.214640 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ff2214_1499_405a_af0d_f397817cc5d9.slice/crio-33023d6c3ec3e3cc1fb0549188dadd762bc695f830e38c7024d19bfffb411bf5 WatchSource:0}: Error finding container 33023d6c3ec3e3cc1fb0549188dadd762bc695f830e38c7024d19bfffb411bf5: Status 404 returned error can't find the container with id 33023d6c3ec3e3cc1fb0549188dadd762bc695f830e38c7024d19bfffb411bf5 Jul 16 10:37:56.608356 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:56.608325 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:56.807482 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:56.807450 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:37:57.124514 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:57.124477 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" event={"ID":"f3ff2214-1499-405a-af0d-f397817cc5d9","Type":"ContainerStarted","Data":"33023d6c3ec3e3cc1fb0549188dadd762bc695f830e38c7024d19bfffb411bf5"} Jul 16 10:37:58.842587 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.842541 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:37:58.842855 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.842636 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:37:58.842855 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.842666 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:37:58.854734 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.854712 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5"] Jul 16 10:37:58.859228 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.859212 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:58.869102 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.869078 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 10:37:58.870737 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.870722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 10:37:58.889281 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.889262 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ght6v\"" Jul 16 10:37:58.889400 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.889329 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 10:37:58.899023 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:58.896377 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5"] Jul 16 10:37:59.037804 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.037762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cck9m\" (UniqueName: \"kubernetes.io/projected/19b1da1f-e048-4802-8b67-7587c279e3ff-kube-api-access-cck9m\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.037804 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.037803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.038035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.037834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b1da1f-e048-4802-8b67-7587c279e3ff-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.038035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.037897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.038035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.037985 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.038132 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.038099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.038165 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.038126 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.135869 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.135785 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" event={"ID":"f3ff2214-1499-405a-af0d-f397817cc5d9","Type":"ContainerStarted","Data":"3db0142a643b4559786f71c64c898f2f2605965c85f22573cb5263e8710bbb8a"} Jul 16 10:37:59.139320 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b1da1f-e048-4802-8b67-7587c279e3ff-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.139401 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139331 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.139401 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.139500 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.139500 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.139601 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cck9m\" (UniqueName: \"kubernetes.io/projected/19b1da1f-e048-4802-8b67-7587c279e3ff-kube-api-access-cck9m\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.139601 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.139524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.140401 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.140371 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.141783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.141754 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b1da1f-e048-4802-8b67-7587c279e3ff-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.141932 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.141872 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.142013 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.141981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.142069 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.142028 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.163105 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.163085 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b1da1f-e048-4802-8b67-7587c279e3ff-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.165471 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.165446 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cck9m\" (UniqueName: \"kubernetes.io/projected/19b1da1f-e048-4802-8b67-7587c279e3ff-kube-api-access-cck9m\") pod \"istiod-openshift-gateway-94bb8fbfd-6nsj5\" (UID: \"19b1da1f-e048-4802-8b67-7587c279e3ff\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.182171 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.182132 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" podStartSLOduration=1.5565058299999999 podStartE2EDuration="4.182119429s" totalImagesPullingTime="2.625613599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:59.180175115 +0000 UTC m=+338.316901787" watchObservedRunningTime="2026-07-16 10:37:59.182119429 +0000 UTC m=+338.318846100" Jul 16 10:37:59.185934 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.185893 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:37:59.370391 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.370279 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5"] Jul 16 10:37:59.373331 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:37:59.373293 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b1da1f_e048_4802_8b67_7587c279e3ff.slice/crio-816d395c371ef94fbbe6106d648a7e8ea04d6bc286dbd956e885c75aa9378c5f WatchSource:0}: Error finding container 816d395c371ef94fbbe6106d648a7e8ea04d6bc286dbd956e885c75aa9378c5f: Status 404 returned error can't find the container with id 816d395c371ef94fbbe6106d648a7e8ea04d6bc286dbd956e885c75aa9378c5f Jul 16 10:37:59.953012 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.952981 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:37:59.958458 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:37:59.958406 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:38:00.142579 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:00.142537 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" event={"ID":"19b1da1f-e048-4802-8b67-7587c279e3ff","Type":"ContainerStarted","Data":"816d395c371ef94fbbe6106d648a7e8ea04d6bc286dbd956e885c75aa9378c5f"} Jul 16 10:38:00.142887 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:00.142869 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:38:00.143803 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:00.143774 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mztsf" Jul 16 10:38:02.140349 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:02.140316 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:38:02.140628 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:02.140378 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:38:02.747052 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:02.747024 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:03.152241 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.152197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:03.157681 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.157650 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" event={"ID":"19b1da1f-e048-4802-8b67-7587c279e3ff","Type":"ContainerStarted","Data":"66eef2cb74775e279611568ecc44764e41b9976856cd5cc3da82299c7bf72ea1"} Jul 16 10:38:03.157861 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.157843 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:38:03.159549 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.159526 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6nsj5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 10:38:03.159670 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.159588 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" podUID="19b1da1f-e048-4802-8b67-7587c279e3ff" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 10:38:03.335966 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.335890 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" podStartSLOduration=2.571012454 podStartE2EDuration="5.335873571s" totalImagesPullingTime="2.764861117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:38:03.314100173 +0000 UTC m=+342.450826845" watchObservedRunningTime="2026-07-16 10:38:03.335873571 +0000 UTC m=+342.472600243" Jul 16 10:38:03.560287 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.560186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:03.673657 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:03.673623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:04.163635 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:04.163606 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6nsj5" Jul 16 10:38:04.744488 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:04.744455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:04.875100 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:04.875069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:17.455447 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:17.455374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:17.554092 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:17.554060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:19.293637 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:19.293600 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:19.395463 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:19.395438 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:45.691229 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:45.691183 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:45.790376 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:45.790336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:45.887489 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:45.887461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:46.639157 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:46.639120 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:38:46.848096 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:38:46.848065 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:15.916315 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:15.916280 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:19.922778 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:19.922749 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:20.786958 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:20.786928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:20.887484 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:20.887453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:22.506969 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:22.506934 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:24.223208 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:24.223179 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:25.353796 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:25.353752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:27.174549 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:27.174517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:27.331467 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:27.331436 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:27.945004 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:27.944975 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:29.765865 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:29.765840 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:29.884056 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:29.884025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:31.725623 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:31.725592 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b7b79598-zj6v5"] Jul 16 10:39:31.987676 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:31.987605 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:32.077594 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:32.077564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:32.191941 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:32.191900 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:32.348387 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:32.348352 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:32.461449 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:32.461421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:34.146885 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:34.146854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:34.275166 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:34.275132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:35.481548 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:35.481519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:36.184467 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:36.184437 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:36.598261 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:36.598233 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:39.301817 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:39.301786 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:39.440097 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:39.440063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:42.941711 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:42.941632 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:43.044457 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:43.044424 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:47.964773 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:47.964741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:48.086815 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:48.086782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:48.212894 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:48.212859 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:50.442569 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:50.442537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:50.546921 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:50.546872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:53.461375 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:53.461339 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:53.572338 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:53.572307 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:56.746152 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:56.746108 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b7b79598-zj6v5" podUID="36e53de7-356e-496c-811c-3515613b0c9b" containerName="console" containerID="cri-o://89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78" gracePeriod=15 Jul 16 10:39:56.992209 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:56.992184 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:39:57.101769 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.101746 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-oauth-config\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.101940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.101807 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-service-ca\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.101940 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.101833 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-trusted-ca-bundle\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.102057 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.101967 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-serving-cert\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.102057 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102023 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-console-config\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.102174 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102056 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ws2r\" (UniqueName: \"kubernetes.io/projected/36e53de7-356e-496c-811c-3515613b0c9b-kube-api-access-8ws2r\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.102174 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102083 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-oauth-serving-cert\") pod \"36e53de7-356e-496c-811c-3515613b0c9b\" (UID: \"36e53de7-356e-496c-811c-3515613b0c9b\") " Jul 16 10:39:57.102340 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102171 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:39:57.102395 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102350 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-service-ca" (OuterVolumeSpecName: "service-ca") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:39:57.102434 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102391 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-trusted-ca-bundle\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.102434 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102417 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-console-config" (OuterVolumeSpecName: "console-config") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:39:57.102624 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.102601 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:39:57.103947 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.103920 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:39:57.104040 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.103961 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:39:57.104040 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.103987 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e53de7-356e-496c-811c-3515613b0c9b-kube-api-access-8ws2r" (OuterVolumeSpecName: "kube-api-access-8ws2r") pod "36e53de7-356e-496c-811c-3515613b0c9b" (UID: "36e53de7-356e-496c-811c-3515613b0c9b"). InnerVolumeSpecName "kube-api-access-8ws2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:57.203789 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.203758 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-console-config\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.203789 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.203784 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ws2r\" (UniqueName: \"kubernetes.io/projected/36e53de7-356e-496c-811c-3515613b0c9b-kube-api-access-8ws2r\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.203789 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.203794 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-oauth-serving-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.204018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.203803 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-oauth-config\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.204018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.203812 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e53de7-356e-496c-811c-3515613b0c9b-service-ca\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.204018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.203819 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e53de7-356e-496c-811c-3515613b0c9b-console-serving-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:39:57.392558 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.392478 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:57.494258 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.494226 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:39:57.656042 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.655964 2568 generic.go:356] "Generic (PLEG): container finished" podID="36e53de7-356e-496c-811c-3515613b0c9b" containerID="89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78" exitCode=0 Jul 16 10:39:57.656042 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.656018 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7b79598-zj6v5" Jul 16 10:39:57.656201 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.656056 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7b79598-zj6v5" event={"ID":"36e53de7-356e-496c-811c-3515613b0c9b","Type":"ContainerDied","Data":"89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78"} Jul 16 10:39:57.656201 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.656108 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7b79598-zj6v5" event={"ID":"36e53de7-356e-496c-811c-3515613b0c9b","Type":"ContainerDied","Data":"be81d92cd95c2569818ccee6f59641ae79e422a3c4f5909143161a69302534f0"} Jul 16 10:39:57.656201 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.656129 2568 scope.go:117] "RemoveContainer" containerID="89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78" Jul 16 10:39:57.664251 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.664228 2568 scope.go:117] "RemoveContainer" containerID="89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78" Jul 16 10:39:57.664506 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:39:57.664482 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78\": container with ID starting with 89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78 not found: ID does not exist" containerID="89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78" Jul 16 10:39:57.664551 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.664519 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78"} err="failed to get container status \"89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78\": rpc error: code = NotFound desc = could not find container \"89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78\": container with ID starting with 89a544362167d6ddeee9f50e0f2fe0813145b6ce09db94c6028d2696ba0f4d78 not found: ID does not exist" Jul 16 10:39:57.688074 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.688053 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b7b79598-zj6v5"] Jul 16 10:39:57.692045 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:57.692020 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b7b79598-zj6v5"] Jul 16 10:39:59.388879 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:39:59.388848 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e53de7-356e-496c-811c-3515613b0c9b" path="/var/lib/kubelet/pods/36e53de7-356e-496c-811c-3515613b0c9b/volumes" Jul 16 10:40:05.983038 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:05.983006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:06.100128 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:06.100086 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:10.581390 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:10.581358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:10.690258 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:10.690234 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:20.271162 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:20.271129 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:20.684347 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:20.684313 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:27.066118 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:27.066081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:27.185621 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:27.185588 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:27.282720 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:27.282685 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:28.030484 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:28.030456 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:28.175663 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:28.175633 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:48.346201 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.346168 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:48.437124 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.437095 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:48.822105 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.822076 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g"] Jul 16 10:40:48.822478 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.822461 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e53de7-356e-496c-811c-3515613b0c9b" containerName="console" Jul 16 10:40:48.822563 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.822481 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e53de7-356e-496c-811c-3515613b0c9b" containerName="console" Jul 16 10:40:48.822563 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.822560 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e53de7-356e-496c-811c-3515613b0c9b" containerName="console" Jul 16 10:40:48.825289 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.825270 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:48.828117 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.828091 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 10:40:48.828222 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.828161 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 16 10:40:48.829028 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.829010 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 10:40:48.829028 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.829024 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-n5s57\"" Jul 16 10:40:48.835404 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.835383 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g"] Jul 16 10:40:48.864825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.864800 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:40:48.939918 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.939887 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3fe50308-0e8d-4430-8ad4-41381e6f8837-data\") pod \"postgres-69fb4594cf-p5m8g\" (UID: \"3fe50308-0e8d-4430-8ad4-41381e6f8837\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:48.940043 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:48.939957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnltv\" (UniqueName: \"kubernetes.io/projected/3fe50308-0e8d-4430-8ad4-41381e6f8837-kube-api-access-cnltv\") pod \"postgres-69fb4594cf-p5m8g\" (UID: \"3fe50308-0e8d-4430-8ad4-41381e6f8837\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:49.040799 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.040772 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3fe50308-0e8d-4430-8ad4-41381e6f8837-data\") pod \"postgres-69fb4594cf-p5m8g\" (UID: \"3fe50308-0e8d-4430-8ad4-41381e6f8837\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:49.040935 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.040801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnltv\" (UniqueName: \"kubernetes.io/projected/3fe50308-0e8d-4430-8ad4-41381e6f8837-kube-api-access-cnltv\") pod \"postgres-69fb4594cf-p5m8g\" (UID: \"3fe50308-0e8d-4430-8ad4-41381e6f8837\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:49.041137 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.041120 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3fe50308-0e8d-4430-8ad4-41381e6f8837-data\") pod \"postgres-69fb4594cf-p5m8g\" (UID: \"3fe50308-0e8d-4430-8ad4-41381e6f8837\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:49.054153 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.054124 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnltv\" (UniqueName: \"kubernetes.io/projected/3fe50308-0e8d-4430-8ad4-41381e6f8837-kube-api-access-cnltv\") pod \"postgres-69fb4594cf-p5m8g\" (UID: \"3fe50308-0e8d-4430-8ad4-41381e6f8837\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:49.136864 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.136812 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:49.276477 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.276448 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g"] Jul 16 10:40:49.278200 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:40:49.278172 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe50308_0e8d_4430_8ad4_41381e6f8837.slice/crio-b6d292ea0e04f48ccbeba8950fce3eee5d4169448e9677b960cb0ecd82f6c60e WatchSource:0}: Error finding container b6d292ea0e04f48ccbeba8950fce3eee5d4169448e9677b960cb0ecd82f6c60e: Status 404 returned error can't find the container with id b6d292ea0e04f48ccbeba8950fce3eee5d4169448e9677b960cb0ecd82f6c60e Jul 16 10:40:49.886975 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:49.886885 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" event={"ID":"3fe50308-0e8d-4430-8ad4-41381e6f8837","Type":"ContainerStarted","Data":"b6d292ea0e04f48ccbeba8950fce3eee5d4169448e9677b960cb0ecd82f6c60e"} Jul 16 10:40:53.907836 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:53.907809 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" event={"ID":"3fe50308-0e8d-4430-8ad4-41381e6f8837","Type":"ContainerStarted","Data":"4f8af84d99bc20beaf9a4bb4a203704c73b619e44d0678bdda61aecf1e611043"} Jul 16 10:40:53.908218 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:53.907877 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:40:53.926881 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:53.926838 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" podStartSLOduration=1.369840207 podStartE2EDuration="5.926824006s" totalImagesPullingTime="4.556983799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:40:53.925578687 +0000 UTC m=+513.062305360" watchObservedRunningTime="2026-07-16 10:40:53.926824006 +0000 UTC m=+513.063550677" Jul 16 10:40:59.942319 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:40:59.942291 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p5m8g" Jul 16 10:41:00.348898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:00.348864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:00.772603 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:00.772576 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:08.012257 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.012197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:08.162034 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.161997 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-b95ms"] Jul 16 10:41:08.178709 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.178681 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.182569 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.182542 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 10:41:08.182897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.182860 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 10:41:08.183320 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.183302 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9j2bz\"" Jul 16 10:41:08.183991 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.183967 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-b95ms"] Jul 16 10:41:08.194296 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.194259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ab5649-dcc0-47e9-8b0d-b831629716b1-webhook-cert\") pod \"maas-controller-6c86f4fc58-b95ms\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.194446 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.194392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t55gz\" (UniqueName: \"kubernetes.io/projected/82ab5649-dcc0-47e9-8b0d-b831629716b1-kube-api-access-t55gz\") pod \"maas-controller-6c86f4fc58-b95ms\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.210581 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.210559 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:08.294901 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.294830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ab5649-dcc0-47e9-8b0d-b831629716b1-webhook-cert\") pod \"maas-controller-6c86f4fc58-b95ms\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.294901 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.294873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t55gz\" (UniqueName: \"kubernetes.io/projected/82ab5649-dcc0-47e9-8b0d-b831629716b1-kube-api-access-t55gz\") pod \"maas-controller-6c86f4fc58-b95ms\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.295666 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.295642 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:08.297391 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.297369 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ab5649-dcc0-47e9-8b0d-b831629716b1-webhook-cert\") pod \"maas-controller-6c86f4fc58-b95ms\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.305729 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.305710 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t55gz\" (UniqueName: \"kubernetes.io/projected/82ab5649-dcc0-47e9-8b0d-b831629716b1-kube-api-access-t55gz\") pod \"maas-controller-6c86f4fc58-b95ms\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.495512 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.495480 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:08.632227 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.632195 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-b95ms"] Jul 16 10:41:08.635630 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:41:08.635604 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ab5649_dcc0_47e9_8b0d_b831629716b1.slice/crio-bd61df28f217e4a19c68f28960a350cbead107c3990e40ae170cfa50d32dfc54 WatchSource:0}: Error finding container bd61df28f217e4a19c68f28960a350cbead107c3990e40ae170cfa50d32dfc54: Status 404 returned error can't find the container with id bd61df28f217e4a19c68f28960a350cbead107c3990e40ae170cfa50d32dfc54 Jul 16 10:41:08.976943 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:08.976843 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" event={"ID":"82ab5649-dcc0-47e9-8b0d-b831629716b1","Type":"ContainerStarted","Data":"bd61df28f217e4a19c68f28960a350cbead107c3990e40ae170cfa50d32dfc54"} Jul 16 10:41:11.127901 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:11.127868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:11.221785 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:11.221752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:11.993089 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:11.993055 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" event={"ID":"82ab5649-dcc0-47e9-8b0d-b831629716b1","Type":"ContainerStarted","Data":"87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91"} Jul 16 10:41:11.993272 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:11.993154 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:12.023413 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:12.023369 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" podStartSLOduration=1.661260558 podStartE2EDuration="4.023355022s" totalImagesPullingTime="2.362094464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:12.021704825 +0000 UTC m=+531.158431499" watchObservedRunningTime="2026-07-16 10:41:12.023355022 +0000 UTC m=+531.160081694" Jul 16 10:41:22.999965 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:22.999866 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:41:42.978429 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:42.978342 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:43.138330 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:43.138294 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:43.488874 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:43.488838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:43.687964 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:43.687928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:44.391234 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:44.391204 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:44.486796 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:44.486767 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:53.509872 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:53.509835 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:41:53.600488 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:41:53.600457 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:02.376437 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:02.376399 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:05.007892 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:05.007863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:13.808456 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.808419 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh"] Jul 16 10:42:13.812405 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.812387 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:13.815161 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.815138 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 10:42:13.816013 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.815994 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 10:42:13.816147 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.816132 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 10:42:13.816232 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.816212 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6m4hr\"" Jul 16 10:42:13.822629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.822602 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh"] Jul 16 10:42:13.906885 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.906854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:13.948786 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.948756 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9574302d-4c3b-4677-b0a0-ffc6fead13a1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:13.948952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.948812 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn2jq\" (UniqueName: \"kubernetes.io/projected/9574302d-4c3b-4677-b0a0-ffc6fead13a1-kube-api-access-jn2jq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:13.948952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.948876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:13.948952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.948921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:13.948952 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.948949 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:13.949087 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:13.949043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050252 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn2jq\" (UniqueName: \"kubernetes.io/projected/9574302d-4c3b-4677-b0a0-ffc6fead13a1-kube-api-access-jn2jq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050414 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050414 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050291 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050414 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050414 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050384 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050599 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9574302d-4c3b-4677-b0a0-ffc6fead13a1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050676 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050655 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050737 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050718 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.050789 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.050741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.052741 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.052719 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9574302d-4c3b-4677-b0a0-ffc6fead13a1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.052877 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.052861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9574302d-4c3b-4677-b0a0-ffc6fead13a1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.059689 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.059642 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn2jq\" (UniqueName: \"kubernetes.io/projected/9574302d-4c3b-4677-b0a0-ffc6fead13a1-kube-api-access-jn2jq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g9sjh\" (UID: \"9574302d-4c3b-4677-b0a0-ffc6fead13a1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.123761 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.123728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:14.258003 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.257970 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh"] Jul 16 10:42:14.260948 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:42:14.260901 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9574302d_4c3b_4677_b0a0_ffc6fead13a1.slice/crio-1d0ed5b76facc4a90df2e8dad5417c60fe106f793231bd313abdf6ccddd72897 WatchSource:0}: Error finding container 1d0ed5b76facc4a90df2e8dad5417c60fe106f793231bd313abdf6ccddd72897: Status 404 returned error can't find the container with id 1d0ed5b76facc4a90df2e8dad5417c60fe106f793231bd313abdf6ccddd72897 Jul 16 10:42:14.270679 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:14.270649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" event={"ID":"9574302d-4c3b-4677-b0a0-ffc6fead13a1","Type":"ContainerStarted","Data":"1d0ed5b76facc4a90df2e8dad5417c60fe106f793231bd313abdf6ccddd72897"} Jul 16 10:42:16.282606 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:16.282568 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" event={"ID":"9574302d-4c3b-4677-b0a0-ffc6fead13a1","Type":"ContainerStarted","Data":"7a97250d8babddc24cb0ee58f4b1c37bcfb3628ac06b35afeabd335684391fc6"} Jul 16 10:42:16.305978 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:16.305930 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" podStartSLOduration=2.241917238 podStartE2EDuration="3.305887898s" totalImagesPullingTime="1.06397066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:42:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:42:16.304090414 +0000 UTC m=+595.440817085" watchObservedRunningTime="2026-07-16 10:42:16.305887898 +0000 UTC m=+595.442614569" Jul 16 10:42:21.246611 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:21.246582 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:42:21.247141 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:21.246843 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:42:21.249314 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:21.249293 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:42:21.249473 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:21.249457 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:42:22.800321 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:22.800294 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:24.124035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:24.124000 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:24.124428 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:24.124086 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:24.136794 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:24.136771 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:24.328624 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:24.328595 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g9sjh" Jul 16 10:42:31.796996 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:31.796964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:35.891310 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:35.891277 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h"] Jul 16 10:42:35.894789 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:35.894771 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:35.897698 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:35.897673 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 10:42:35.908460 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:35.908433 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h"] Jul 16 10:42:35.935023 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:35.934991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:36.030649 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.030618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.030806 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.030685 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.030806 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.030715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.030806 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.030739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.030806 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.030761 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.030806 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.030784 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfpr\" (UniqueName: \"kubernetes.io/projected/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-kube-api-access-4sfpr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132015 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.131975 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfpr\" (UniqueName: \"kubernetes.io/projected/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-kube-api-access-4sfpr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132413 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132356 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132485 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132427 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.132567 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.132543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.134516 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.134491 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.134608 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.134553 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.148050 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.147996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfpr\" (UniqueName: \"kubernetes.io/projected/73a566f2-ff23-40fb-bf4a-9b3e3cc63c59-kube-api-access-4sfpr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fdv7h\" (UID: \"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.204963 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.204931 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:36.351587 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.351563 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h"] Jul 16 10:42:36.353446 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:42:36.353402 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a566f2_ff23_40fb_bf4a_9b3e3cc63c59.slice/crio-96ea691cef3422b372399d4ff5516fd95904d8a3b38a42b57309bd258129bffa WatchSource:0}: Error finding container 96ea691cef3422b372399d4ff5516fd95904d8a3b38a42b57309bd258129bffa: Status 404 returned error can't find the container with id 96ea691cef3422b372399d4ff5516fd95904d8a3b38a42b57309bd258129bffa Jul 16 10:42:36.354746 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.354729 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 10:42:36.373342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:36.373315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" event={"ID":"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59","Type":"ContainerStarted","Data":"96ea691cef3422b372399d4ff5516fd95904d8a3b38a42b57309bd258129bffa"} Jul 16 10:42:37.379536 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:37.379504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" event={"ID":"73a566f2-ff23-40fb-bf4a-9b3e3cc63c59","Type":"ContainerStarted","Data":"bd400368b473fac24338f0410b06cc604ed747191819b800c330653de73167a1"} Jul 16 10:42:37.403703 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:37.403650 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" podStartSLOduration=2.197030212 podStartE2EDuration="2.403631688s" totalImagesPullingTime="206.601476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:42:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:42:37.401780991 +0000 UTC m=+616.538507663" watchObservedRunningTime="2026-07-16 10:42:37.403631688 +0000 UTC m=+616.540358363" Jul 16 10:42:40.506483 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:40.506451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:42:46.205666 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:46.205581 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:46.205666 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:46.205637 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:46.218373 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:46.218350 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:46.428828 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:46.428794 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fdv7h" Jul 16 10:42:54.853528 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:42:54.853496 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:43:22.060873 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:43:22.060827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:43:23.579321 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:43:23.579290 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:43:23.682100 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:43:23.682069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:44:00.587079 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:44:00.587045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:44:00.701038 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:44:00.701004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:44:01.843722 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:44:01.843691 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:44:01.950699 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:44:01.950665 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:44:24.980744 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:44:24.980668 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:44:25.070245 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:44:25.070220 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:45:00.207670 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:00.207633 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h"] Jul 16 10:45:00.211148 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:00.211131 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" Jul 16 10:45:00.215726 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:00.215707 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlcx9\"" Jul 16 10:45:00.220477 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:00.220457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" Jul 16 10:45:00.238585 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:00.238526 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h"] Jul 16 10:45:00.358849 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:00.358818 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h"] Jul 16 10:45:00.361896 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:45:00.361870 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f18848e_74e0_4488_8d43_8e7a9c866fc6.slice/crio-c29b16f25d6e3b69f19ab092295cc1adc0e8304881c70dffaf922db8da91a4e7 WatchSource:0}: Error finding container c29b16f25d6e3b69f19ab092295cc1adc0e8304881c70dffaf922db8da91a4e7: Status 404 returned error can't find the container with id c29b16f25d6e3b69f19ab092295cc1adc0e8304881c70dffaf922db8da91a4e7 Jul 16 10:45:01.034104 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:01.034062 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" event={"ID":"6f18848e-74e0-4488-8d43-8e7a9c866fc6","Type":"ContainerStarted","Data":"c29b16f25d6e3b69f19ab092295cc1adc0e8304881c70dffaf922db8da91a4e7"} Jul 16 10:45:03.046203 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:03.046175 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" event={"ID":"6f18848e-74e0-4488-8d43-8e7a9c866fc6","Type":"ContainerStarted","Data":"e3d3dd1f6c016cc265f44ebb2bc47ef95314bd42f708055b71b35b7b19b8587c"} Jul 16 10:45:03.071602 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:03.071554 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" podStartSLOduration=0.971616784 podStartE2EDuration="3.071535334s" totalImagesPullingTime="2.09991855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:45:03.070973443 +0000 UTC m=+762.207700116" watchObservedRunningTime="2026-07-16 10:45:03.071535334 +0000 UTC m=+762.208262006" Jul 16 10:45:04.052250 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:04.052213 2568 generic.go:356] "Generic (PLEG): container finished" podID="6f18848e-74e0-4488-8d43-8e7a9c866fc6" containerID="e3d3dd1f6c016cc265f44ebb2bc47ef95314bd42f708055b71b35b7b19b8587c" exitCode=0 Jul 16 10:45:04.052616 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:04.052265 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" event={"ID":"6f18848e-74e0-4488-8d43-8e7a9c866fc6","Type":"ContainerDied","Data":"e3d3dd1f6c016cc265f44ebb2bc47ef95314bd42f708055b71b35b7b19b8587c"} Jul 16 10:45:05.192939 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:05.192898 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" Jul 16 10:45:06.062578 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:06.062553 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" Jul 16 10:45:06.062757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:06.062597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736645-rbj4h" event={"ID":"6f18848e-74e0-4488-8d43-8e7a9c866fc6","Type":"ContainerDied","Data":"c29b16f25d6e3b69f19ab092295cc1adc0e8304881c70dffaf922db8da91a4e7"} Jul 16 10:45:06.062757 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:06.062653 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c29b16f25d6e3b69f19ab092295cc1adc0e8304881c70dffaf922db8da91a4e7" Jul 16 10:45:13.744983 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:13.744949 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:45:13.850859 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:13.850824 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-b95ms"] Jul 16 10:45:13.851136 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:13.851095 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" podUID="82ab5649-dcc0-47e9-8b0d-b831629716b1" containerName="manager" containerID="cri-o://87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91" gracePeriod=30 Jul 16 10:45:14.095366 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.095339 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:45:14.101836 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.101812 2568 generic.go:356] "Generic (PLEG): container finished" podID="82ab5649-dcc0-47e9-8b0d-b831629716b1" containerID="87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91" exitCode=0 Jul 16 10:45:14.101958 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.101846 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" Jul 16 10:45:14.101958 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.101863 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" event={"ID":"82ab5649-dcc0-47e9-8b0d-b831629716b1","Type":"ContainerDied","Data":"87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91"} Jul 16 10:45:14.101958 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.101933 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c86f4fc58-b95ms" event={"ID":"82ab5649-dcc0-47e9-8b0d-b831629716b1","Type":"ContainerDied","Data":"bd61df28f217e4a19c68f28960a350cbead107c3990e40ae170cfa50d32dfc54"} Jul 16 10:45:14.101958 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.101958 2568 scope.go:117] "RemoveContainer" containerID="87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91" Jul 16 10:45:14.109783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.109767 2568 scope.go:117] "RemoveContainer" containerID="87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91" Jul 16 10:45:14.110052 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:45:14.110029 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91\": container with ID starting with 87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91 not found: ID does not exist" containerID="87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91" Jul 16 10:45:14.110112 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.110060 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91"} err="failed to get container status \"87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91\": rpc error: code = NotFound desc = could not find container \"87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91\": container with ID starting with 87214587280c57945dd71df3b5eb328d8ded47b2455c37eeb4a4043952607b91 not found: ID does not exist" Jul 16 10:45:14.155748 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.155722 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t55gz\" (UniqueName: \"kubernetes.io/projected/82ab5649-dcc0-47e9-8b0d-b831629716b1-kube-api-access-t55gz\") pod \"82ab5649-dcc0-47e9-8b0d-b831629716b1\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " Jul 16 10:45:14.155862 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.155802 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ab5649-dcc0-47e9-8b0d-b831629716b1-webhook-cert\") pod \"82ab5649-dcc0-47e9-8b0d-b831629716b1\" (UID: \"82ab5649-dcc0-47e9-8b0d-b831629716b1\") " Jul 16 10:45:14.157840 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.157810 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ab5649-dcc0-47e9-8b0d-b831629716b1-kube-api-access-t55gz" (OuterVolumeSpecName: "kube-api-access-t55gz") pod "82ab5649-dcc0-47e9-8b0d-b831629716b1" (UID: "82ab5649-dcc0-47e9-8b0d-b831629716b1"). InnerVolumeSpecName "kube-api-access-t55gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:45:14.157962 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.157851 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ab5649-dcc0-47e9-8b0d-b831629716b1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "82ab5649-dcc0-47e9-8b0d-b831629716b1" (UID: "82ab5649-dcc0-47e9-8b0d-b831629716b1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:45:14.256563 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.256529 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t55gz\" (UniqueName: \"kubernetes.io/projected/82ab5649-dcc0-47e9-8b0d-b831629716b1-kube-api-access-t55gz\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:45:14.256563 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.256571 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ab5649-dcc0-47e9-8b0d-b831629716b1-webhook-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:45:14.425303 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.425269 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-b95ms"] Jul 16 10:45:14.427975 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:14.427952 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-b95ms"] Jul 16 10:45:15.135029 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:15.134995 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:45:15.388056 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:15.387982 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ab5649-dcc0-47e9-8b0d-b831629716b1" path="/var/lib/kubelet/pods/82ab5649-dcc0-47e9-8b0d-b831629716b1/volumes" Jul 16 10:45:56.426934 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:56.426825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:45:56.530532 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:56.530502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:45:56.684160 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:45:56.684092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:02.870661 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:02.870627 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:03.248341 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:03.248260 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:03.349390 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:03.349357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:10.122085 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:10.122050 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:10.214061 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:10.214028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:41.283696 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:41.283664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:46:41.384891 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:46:41.384858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:47:21.275107 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:47:21.275081 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:47:21.275580 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:47:21.275552 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:47:21.277772 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:47:21.277752 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:47:21.278080 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:47:21.278065 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:47:45.731992 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:47:45.731960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:47:47.827470 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:47:47.827439 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:49:08.590375 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:49:08.590285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:49:40.771451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:49:40.771419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:49:40.838557 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:49:40.838530 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:50:19.119372 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:50:19.119296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:52:21.302678 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:52:21.302649 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:52:21.304018 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:52:21.303997 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:52:21.305316 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:52:21.305293 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:52:21.306727 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:52:21.306709 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:54:34.489731 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:54:34.489681 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:56:24.914215 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:56:24.914110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:56:28.983535 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:56:28.983502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:56:39.615488 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:56:39.615455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:08.332555 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:08.332517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:21.332664 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:21.332634 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:57:21.335185 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:21.335161 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:57:21.335764 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:21.335743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 10:57:21.338328 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:21.338312 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 10:57:25.190993 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:25.190954 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:33.469674 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:33.469640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:38.587584 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:38.587550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:41.578087 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:41.578046 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:41.691844 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:41.691812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:42.129995 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:42.129964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:47.210955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:47.210856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:47.310687 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:47.310654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:48.380659 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.380615 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:48.451323 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.451296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:48.574859 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.574833 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6"] Jul 16 10:57:48.575235 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.575222 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ab5649-dcc0-47e9-8b0d-b831629716b1" containerName="manager" Jul 16 10:57:48.575342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.575237 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ab5649-dcc0-47e9-8b0d-b831629716b1" containerName="manager" Jul 16 10:57:48.575342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.575259 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f18848e-74e0-4488-8d43-8e7a9c866fc6" containerName="cleanup" Jul 16 10:57:48.575342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.575266 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f18848e-74e0-4488-8d43-8e7a9c866fc6" containerName="cleanup" Jul 16 10:57:48.575342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.575332 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f18848e-74e0-4488-8d43-8e7a9c866fc6" containerName="cleanup" Jul 16 10:57:48.575342 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.575344 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ab5649-dcc0-47e9-8b0d-b831629716b1" containerName="manager" Jul 16 10:57:48.578346 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.578326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.581679 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.581661 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-eb5cfebe-openshift-default-dockercfg-jzq9s\"" Jul 16 10:57:48.608359 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.608328 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6"] Jul 16 10:57:48.615164 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.615142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:48.707093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707014 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-credential-socket\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-token\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-socket\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707368 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjqpl\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-kube-api-access-vjqpl\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707368 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707239 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-envoy\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707368 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-podinfo\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707368 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0226ac50-f0db-4a9a-9e81-e145a1343b03-istiod-ca-cert\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707368 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-data\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.707549 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.707369 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-certs\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.712254 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.712196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:48.808276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808245 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqpl\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-kube-api-access-vjqpl\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.808467 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-envoy\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.808467 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-podinfo\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.808467 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808351 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0226ac50-f0db-4a9a-9e81-e145a1343b03-istiod-ca-cert\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.808467 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-data\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.808686 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-certs\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.808746 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-credential-socket\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809008 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.808973 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-data\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809132 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809033 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-credential-socket\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809132 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-token\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809132 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-socket\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809306 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-certs\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809528 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809479 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-socket\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809694 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809620 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0226ac50-f0db-4a9a-9e81-e145a1343b03-istiod-ca-cert\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.809803 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.809675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:48.810695 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.810673 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-envoy\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.811012 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.810995 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-podinfo\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.826243 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.826215 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqpl\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-kube-api-access-vjqpl\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.826821 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.826788 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-token\") pod \"e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.889311 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.889281 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:48.929380 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:48.929351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:49.031626 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.031602 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6"] Jul 16 10:57:49.033519 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:57:49.033498 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0226ac50_f0db_4a9a_9e81_e145a1343b03.slice/crio-b464c107cab6b5323078f07d1b07bfa4e39053a6a600d36c51927abf566a5d39 WatchSource:0}: Error finding container b464c107cab6b5323078f07d1b07bfa4e39053a6a600d36c51927abf566a5d39: Status 404 returned error can't find the container with id b464c107cab6b5323078f07d1b07bfa4e39053a6a600d36c51927abf566a5d39 Jul 16 10:57:49.035603 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.035578 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:57:49.035658 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.035646 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:57:49.035696 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.035675 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:57:49.539589 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.539549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" event={"ID":"0226ac50-f0db-4a9a-9e81-e145a1343b03","Type":"ContainerStarted","Data":"480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a"} Jul 16 10:57:49.540014 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.539608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" event={"ID":"0226ac50-f0db-4a9a-9e81-e145a1343b03","Type":"ContainerStarted","Data":"b464c107cab6b5323078f07d1b07bfa4e39053a6a600d36c51927abf566a5d39"} Jul 16 10:57:49.564354 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.564311 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" podStartSLOduration=1.564295929 podStartE2EDuration="1.564295929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:57:49.562728171 +0000 UTC m=+1528.699454845" watchObservedRunningTime="2026-07-16 10:57:49.564295929 +0000 UTC m=+1528.701022600" Jul 16 10:57:49.567305 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.567279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:49.670326 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.670277 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:49.890371 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:49.890342 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:50.894597 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:50.894569 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:51.549886 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:51.549855 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:51.550713 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:51.550691 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:57:53.075163 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:53.075124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:54.888469 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:54.888437 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6"] Jul 16 10:57:54.888858 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:54.888621 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" podUID="0226ac50-f0db-4a9a-9e81-e145a1343b03" containerName="istio-proxy" containerID="cri-o://480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a" gracePeriod=30 Jul 16 10:57:54.897179 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:54.897154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:55.012576 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:55.012543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:57:56.846291 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:57:56.846256 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:00.134613 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.134588 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:58:00.209607 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209528 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-podinfo\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.209607 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209563 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-token\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.209823 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209658 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-credential-socket\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.209823 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209703 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-certs\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.209823 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209727 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0226ac50-f0db-4a9a-9e81-e145a1343b03-istiod-ca-cert\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.209823 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209755 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-data\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.209823 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209800 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-socket\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.210110 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209837 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-envoy\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.210110 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.209961 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:00.210110 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210008 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:00.210276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210106 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-data" (OuterVolumeSpecName: "istio-data") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:00.210276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210152 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:00.210276 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210261 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.210425 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210279 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.210425 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210295 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.210425 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210308 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.210587 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.210460 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0226ac50-f0db-4a9a-9e81-e145a1343b03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:58:00.211856 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.211823 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 10:58:00.211987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.211837 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-token" (OuterVolumeSpecName: "istio-token") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:58:00.211987 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.211869 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:00.310898 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.310864 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjqpl\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-kube-api-access-vjqpl\") pod \"0226ac50-f0db-4a9a-9e81-e145a1343b03\" (UID: \"0226ac50-f0db-4a9a-9e81-e145a1343b03\") " Jul 16 10:58:00.311082 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.311075 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.311124 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.311087 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.311124 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.311096 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0226ac50-f0db-4a9a-9e81-e145a1343b03-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.311124 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.311104 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0226ac50-f0db-4a9a-9e81-e145a1343b03-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.312829 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.312807 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-kube-api-access-vjqpl" (OuterVolumeSpecName: "kube-api-access-vjqpl") pod "0226ac50-f0db-4a9a-9e81-e145a1343b03" (UID: "0226ac50-f0db-4a9a-9e81-e145a1343b03"). InnerVolumeSpecName "kube-api-access-vjqpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:58:00.411511 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.411476 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjqpl\" (UniqueName: \"kubernetes.io/projected/0226ac50-f0db-4a9a-9e81-e145a1343b03-kube-api-access-vjqpl\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:00.592427 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.592390 2568 generic.go:356] "Generic (PLEG): container finished" podID="0226ac50-f0db-4a9a-9e81-e145a1343b03" containerID="480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a" exitCode=0 Jul 16 10:58:00.592427 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.592430 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" event={"ID":"0226ac50-f0db-4a9a-9e81-e145a1343b03","Type":"ContainerDied","Data":"480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a"} Jul 16 10:58:00.592629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.592471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" event={"ID":"0226ac50-f0db-4a9a-9e81-e145a1343b03","Type":"ContainerDied","Data":"b464c107cab6b5323078f07d1b07bfa4e39053a6a600d36c51927abf566a5d39"} Jul 16 10:58:00.592629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.592494 2568 scope.go:117] "RemoveContainer" containerID="480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a" Jul 16 10:58:00.592629 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.592432 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6" Jul 16 10:58:00.601345 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.601320 2568 scope.go:117] "RemoveContainer" containerID="480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a" Jul 16 10:58:00.601596 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:58:00.601580 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a\": container with ID starting with 480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a not found: ID does not exist" containerID="480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a" Jul 16 10:58:00.601634 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.601606 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a"} err="failed to get container status \"480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a\": rpc error: code = NotFound desc = could not find container \"480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a\": container with ID starting with 480efda88f7f14aebeffdbaef248f5c50479c484ea4fab8af0411c8739a4149a not found: ID does not exist" Jul 16 10:58:00.618980 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.618955 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6"] Jul 16 10:58:00.622367 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:00.622349 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-eb5cfebe-openshift-default-75599bffbf-bpxh6"] Jul 16 10:58:01.332219 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:01.332180 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:01.392140 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:01.392108 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0226ac50-f0db-4a9a-9e81-e145a1343b03" path="/var/lib/kubelet/pods/0226ac50-f0db-4a9a-9e81-e145a1343b03/volumes" Jul 16 10:58:01.459895 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:01.459863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.231239 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.231212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.301723 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.301693 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.516767 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.516683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.620538 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.620508 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.687049 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.687005 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt"] Jul 16 10:58:07.688945 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.688024 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0226ac50-f0db-4a9a-9e81-e145a1343b03" containerName="istio-proxy" Jul 16 10:58:07.688945 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.688046 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0226ac50-f0db-4a9a-9e81-e145a1343b03" containerName="istio-proxy" Jul 16 10:58:07.688945 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.688244 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0226ac50-f0db-4a9a-9e81-e145a1343b03" containerName="istio-proxy" Jul 16 10:58:07.694275 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.694249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.700278 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.700253 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-3eb28edb-gw-openshift-default-dockercfg-sx2w8\"" Jul 16 10:58:07.717054 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.717029 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt"] Jul 16 10:58:07.774055 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.773981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-envoy\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774055 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt7r5\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-kube-api-access-xt7r5\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774235 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-credential-socket\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774235 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774146 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-certs\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774235 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-istio-token\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774235 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de6d8184-7783-4898-a563-18f70bc344ba-istiod-ca-cert\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774419 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-data\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774419 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774317 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de6d8184-7783-4898-a563-18f70bc344ba-istio-podinfo\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.774419 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.774373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-socket\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.776019 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.775998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.871510 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.871478 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:07.875661 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-envoy\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.875783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875701 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt7r5\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-kube-api-access-xt7r5\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.875783 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-credential-socket\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.875897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875796 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-certs\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.875897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-istio-token\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.875897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de6d8184-7783-4898-a563-18f70bc344ba-istiod-ca-cert\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.875897 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.875888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-data\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876175 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876144 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de6d8184-7783-4898-a563-18f70bc344ba-istio-podinfo\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-socket\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-credential-socket\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876267 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-data\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876313 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-certs\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876529 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876505 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-socket\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.876686 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.876667 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de6d8184-7783-4898-a563-18f70bc344ba-istiod-ca-cert\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.877934 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.877897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-envoy\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.878494 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.878472 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de6d8184-7783-4898-a563-18f70bc344ba-istio-podinfo\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.894493 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.894471 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-istio-token\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:07.894973 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:07.894953 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt7r5\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-kube-api-access-xt7r5\") pod \"e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:08.006087 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.006051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:08.104553 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.104503 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:08.152352 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.152320 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt"] Jul 16 10:58:08.155701 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:58:08.155676 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6d8184_7783_4898_a563_18f70bc344ba.slice/crio-52de008fb5a77fe958cf008d40c08a7846df1abe47f789f66f1ff10940d1073d WatchSource:0}: Error finding container 52de008fb5a77fe958cf008d40c08a7846df1abe47f789f66f1ff10940d1073d: Status 404 returned error can't find the container with id 52de008fb5a77fe958cf008d40c08a7846df1abe47f789f66f1ff10940d1073d Jul 16 10:58:08.157736 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.157706 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:58:08.157793 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.157772 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:58:08.157825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.157802 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:58:08.222982 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.222959 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:08.599144 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.599113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:08.629741 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.629713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" event={"ID":"de6d8184-7783-4898-a563-18f70bc344ba","Type":"ContainerStarted","Data":"654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6"} Jul 16 10:58:08.629879 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.629760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" event={"ID":"de6d8184-7783-4898-a563-18f70bc344ba","Type":"ContainerStarted","Data":"52de008fb5a77fe958cf008d40c08a7846df1abe47f789f66f1ff10940d1073d"} Jul 16 10:58:08.657220 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.657171 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" podStartSLOduration=1.657158018 podStartE2EDuration="1.657158018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:58:08.655180828 +0000 UTC m=+1547.791907535" watchObservedRunningTime="2026-07-16 10:58:08.657158018 +0000 UTC m=+1547.793884690" Jul 16 10:58:08.777250 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.777216 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:08.874549 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:08.874460 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:09.006675 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:09.006647 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:09.011462 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:09.011438 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:09.635101 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:09.635070 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:09.636044 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:09.636022 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:11.116074 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:11.116042 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:11.218161 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:11.218131 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:13.604441 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:13.604403 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt"] Jul 16 10:58:13.605007 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:13.604623 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" podUID="de6d8184-7783-4898-a563-18f70bc344ba" containerName="istio-proxy" containerID="cri-o://654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6" gracePeriod=30 Jul 16 10:58:13.631063 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:13.631031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:13.731084 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:13.731053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:14.990965 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:14.990897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:19.085035 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:19.084999 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:20.067273 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.067248 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:20.251028 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.251005 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:20.395793 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.395767 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-socket\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.395972 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.395803 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-certs\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.395972 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.395833 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-data\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.395972 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.395859 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de6d8184-7783-4898-a563-18f70bc344ba-istio-podinfo\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.395972 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.395930 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt7r5\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-kube-api-access-xt7r5\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.396181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396013 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-credential-socket\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.396181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396041 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-istio-token\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.396181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396066 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-envoy\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.396181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396095 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de6d8184-7783-4898-a563-18f70bc344ba-istiod-ca-cert\") pod \"de6d8184-7783-4898-a563-18f70bc344ba\" (UID: \"de6d8184-7783-4898-a563-18f70bc344ba\") " Jul 16 10:58:20.396181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396128 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:20.396181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396130 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:20.396491 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396314 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.396491 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396317 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:20.396491 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396332 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.396491 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396471 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6d8184-7783-4898-a563-18f70bc344ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:58:20.396742 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.396712 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:20.398223 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.398187 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de6d8184-7783-4898-a563-18f70bc344ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 10:58:20.398318 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.398253 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:58:20.398318 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.398267 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:20.398318 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.398301 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-kube-api-access-xt7r5" (OuterVolumeSpecName: "kube-api-access-xt7r5") pod "de6d8184-7783-4898-a563-18f70bc344ba" (UID: "de6d8184-7783-4898-a563-18f70bc344ba"). InnerVolumeSpecName "kube-api-access-xt7r5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:58:20.497421 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497394 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.497421 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497418 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de6d8184-7783-4898-a563-18f70bc344ba-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.497576 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497433 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt7r5\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-kube-api-access-xt7r5\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.497576 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497447 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.497576 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497458 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de6d8184-7783-4898-a563-18f70bc344ba-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.497576 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497470 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de6d8184-7783-4898-a563-18f70bc344ba-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.497576 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.497481 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de6d8184-7783-4898-a563-18f70bc344ba-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:20.698264 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.698176 2568 generic.go:356] "Generic (PLEG): container finished" podID="de6d8184-7783-4898-a563-18f70bc344ba" containerID="654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6" exitCode=0 Jul 16 10:58:20.698264 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.698227 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" Jul 16 10:58:20.698264 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.698240 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" event={"ID":"de6d8184-7783-4898-a563-18f70bc344ba","Type":"ContainerDied","Data":"654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6"} Jul 16 10:58:20.698465 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.698319 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt" event={"ID":"de6d8184-7783-4898-a563-18f70bc344ba","Type":"ContainerDied","Data":"52de008fb5a77fe958cf008d40c08a7846df1abe47f789f66f1ff10940d1073d"} Jul 16 10:58:20.698465 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.698348 2568 scope.go:117] "RemoveContainer" containerID="654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6" Jul 16 10:58:20.707480 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.707464 2568 scope.go:117] "RemoveContainer" containerID="654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6" Jul 16 10:58:20.707712 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:58:20.707693 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6\": container with ID starting with 654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6 not found: ID does not exist" containerID="654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6" Jul 16 10:58:20.707755 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.707718 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6"} err="failed to get container status \"654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6\": rpc error: code = NotFound desc = could not find container \"654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6\": container with ID starting with 654666ae4c49903289a9ba52ae57eb684ccdcb8574562dbcb40135521a3850d6 not found: ID does not exist" Jul 16 10:58:20.721725 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.721701 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt"] Jul 16 10:58:20.731031 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:20.731008 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-3eb28edb-gw-openshift-default-684664f9b7-mptgt"] Jul 16 10:58:21.389212 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:21.389133 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de6d8184-7783-4898-a563-18f70bc344ba" path="/var/lib/kubelet/pods/de6d8184-7783-4898-a563-18f70bc344ba/volumes" Jul 16 10:58:25.918243 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:25.918209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:26.018213 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.018183 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:26.119627 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.119594 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t"] Jul 16 10:58:26.120070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.120010 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6d8184-7783-4898-a563-18f70bc344ba" containerName="istio-proxy" Jul 16 10:58:26.120070 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.120024 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6d8184-7783-4898-a563-18f70bc344ba" containerName="istio-proxy" Jul 16 10:58:26.120178 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.120115 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6d8184-7783-4898-a563-18f70bc344ba" containerName="istio-proxy" Jul 16 10:58:26.123106 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.123087 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.125764 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.125746 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-95aa3181-openshift-default-dockercfg-dqccr\"" Jul 16 10:58:26.140593 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.140569 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t"] Jul 16 10:58:26.149558 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.149540 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:26.244452 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-credential-socket\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244452 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g56cw\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-kube-api-access-g56cw\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244653 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-certs\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244653 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-envoy\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244653 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-data\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244653 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istiod-ca-cert\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-socket\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-podinfo\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.244825 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.244712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-token\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.245843 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.245818 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:26.345215 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345189 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-certs\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345215 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-envoy\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-data\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istiod-ca-cert\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-socket\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345340 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:26.345435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-podinfo\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345435 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-token\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345761 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-credential-socket\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345761 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g56cw\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-kube-api-access-g56cw\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.345761 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345670 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-data\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.346002 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.345971 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-credential-socket\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.346141 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.346065 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-socket\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.346218 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.346138 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-certs\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.346331 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.346307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istiod-ca-cert\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.348206 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.348173 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-podinfo\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.348507 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.348489 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-envoy\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.360226 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.360201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-token\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.362655 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.362623 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g56cw\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-kube-api-access-g56cw\") pod \"e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.435422 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.435396 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:26.439223 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.439190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:26.575287 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.575256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t"] Jul 16 10:58:26.577480 ip-10-0-138-186 kubenswrapper[2568]: W0716 10:58:26.577456 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220f2921_5ca5_4285_ac0c_5b1fb8750ee2.slice/crio-ae34826b2e3486369da19e8d01ac5c7d8ebce947f6ebf5a6d922e85d70b7e64c WatchSource:0}: Error finding container ae34826b2e3486369da19e8d01ac5c7d8ebce947f6ebf5a6d922e85d70b7e64c: Status 404 returned error can't find the container with id ae34826b2e3486369da19e8d01ac5c7d8ebce947f6ebf5a6d922e85d70b7e64c Jul 16 10:58:26.579379 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.579351 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:58:26.579438 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.579411 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:58:26.579473 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.579441 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 10:58:26.727975 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.727939 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" event={"ID":"220f2921-5ca5-4285-ac0c-5b1fb8750ee2","Type":"ContainerStarted","Data":"fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9"} Jul 16 10:58:26.728154 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.727993 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" event={"ID":"220f2921-5ca5-4285-ac0c-5b1fb8750ee2","Type":"ContainerStarted","Data":"ae34826b2e3486369da19e8d01ac5c7d8ebce947f6ebf5a6d922e85d70b7e64c"} Jul 16 10:58:26.793569 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:26.793474 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" podStartSLOduration=0.793458029 podStartE2EDuration="793.458029ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:58:26.791358255 +0000 UTC m=+1565.928084926" watchObservedRunningTime="2026-07-16 10:58:26.793458029 +0000 UTC m=+1565.930184701" Jul 16 10:58:27.091009 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:27.090975 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:27.191821 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:27.191791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:27.436573 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:27.436493 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:27.441451 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:27.441423 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:27.734041 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:27.733954 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:27.734808 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:27.734791 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:29.591607 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:29.591581 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:31.803057 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:31.803023 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t"] Jul 16 10:58:31.803472 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:31.803204 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" podUID="220f2921-5ca5-4285-ac0c-5b1fb8750ee2" containerName="istio-proxy" containerID="cri-o://fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9" gracePeriod=30 Jul 16 10:58:31.835295 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:31.835263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:31.935551 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:31.935520 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:33.211087 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:33.211052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:37.051935 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.051899 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:37.238093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238059 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-podinfo\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238093 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238098 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-data\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238307 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238130 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-socket\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238307 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238159 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g56cw\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-kube-api-access-g56cw\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238307 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238216 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-certs\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238307 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238299 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-token\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238538 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238327 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istiod-ca-cert\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238538 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238360 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-envoy\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238538 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238359 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-data" (OuterVolumeSpecName: "istio-data") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:37.238538 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238439 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-credential-socket\") pod \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\" (UID: \"220f2921-5ca5-4285-ac0c-5b1fb8750ee2\") " Jul 16 10:58:37.238734 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238534 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:37.238734 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238564 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:37.238841 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238767 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:37.238841 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238780 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.238841 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238801 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.238841 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238806 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:58:37.239021 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.238819 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.240402 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.240378 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 10:58:37.240514 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.240432 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-token" (OuterVolumeSpecName: "istio-token") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:58:37.240584 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.240565 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-kube-api-access-g56cw" (OuterVolumeSpecName: "kube-api-access-g56cw") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "kube-api-access-g56cw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:58:37.240636 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.240620 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "220f2921-5ca5-4285-ac0c-5b1fb8750ee2" (UID: "220f2921-5ca5-4285-ac0c-5b1fb8750ee2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:58:37.340237 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.340212 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.340345 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.340252 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.340345 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.340261 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.340345 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.340270 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.340345 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.340280 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.340345 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.340288 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g56cw\" (UniqueName: \"kubernetes.io/projected/220f2921-5ca5-4285-ac0c-5b1fb8750ee2-kube-api-access-g56cw\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 10:58:37.438955 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.438922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:37.780681 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.780647 2568 generic.go:356] "Generic (PLEG): container finished" podID="220f2921-5ca5-4285-ac0c-5b1fb8750ee2" containerID="fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9" exitCode=0 Jul 16 10:58:37.780821 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.780689 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" Jul 16 10:58:37.780821 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.780720 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" event={"ID":"220f2921-5ca5-4285-ac0c-5b1fb8750ee2","Type":"ContainerDied","Data":"fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9"} Jul 16 10:58:37.780821 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.780761 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t" event={"ID":"220f2921-5ca5-4285-ac0c-5b1fb8750ee2","Type":"ContainerDied","Data":"ae34826b2e3486369da19e8d01ac5c7d8ebce947f6ebf5a6d922e85d70b7e64c"} Jul 16 10:58:37.780821 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.780783 2568 scope.go:117] "RemoveContainer" containerID="fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9" Jul 16 10:58:37.789390 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.789368 2568 scope.go:117] "RemoveContainer" containerID="fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9" Jul 16 10:58:37.789647 ip-10-0-138-186 kubenswrapper[2568]: E0716 10:58:37.789627 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9\": container with ID starting with fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9 not found: ID does not exist" containerID="fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9" Jul 16 10:58:37.789723 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.789660 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9"} err="failed to get container status \"fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9\": rpc error: code = NotFound desc = could not find container \"fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9\": container with ID starting with fa8625d4565977c58f9995870c6c3dc0a8b36dd955b7611d4442ec952210c5f9 not found: ID does not exist" Jul 16 10:58:37.799856 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.799835 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t"] Jul 16 10:58:37.802631 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:37.802611 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-95aa3181-openshift-default-bcbb586f5-zn96t"] Jul 16 10:58:38.395738 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:38.395711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:39.391428 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:39.391396 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220f2921-5ca5-4285-ac0c-5b1fb8750ee2" path="/var/lib/kubelet/pods/220f2921-5ca5-4285-ac0c-5b1fb8750ee2/volumes" Jul 16 10:58:44.938073 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:44.938031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:45.099840 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:45.099809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:45.450589 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:45.450558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:45.582956 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:45.582902 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:47.086846 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:47.086813 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:53.819181 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:53.819146 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:53.943722 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:53.943686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:54.068020 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:54.067990 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:58:54.547667 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:58:54.547636 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:20.514803 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:20.514730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:26.831746 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:26.831713 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:26.916487 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:26.916455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:27.027470 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:27.027436 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:27.126721 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:27.126632 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:42.987267 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:42.987234 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:49.192649 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:49.192614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:49.260925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:49.260867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:49.360925 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:49.360878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:49.458444 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:49.458355 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 10:59:59.936045 ip-10-0-138-186 kubenswrapper[2568]: I0716 10:59:59.936016 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:00.152066 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.152031 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq"] Jul 16 11:00:00.152393 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.152380 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220f2921-5ca5-4285-ac0c-5b1fb8750ee2" containerName="istio-proxy" Jul 16 11:00:00.152393 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.152394 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="220f2921-5ca5-4285-ac0c-5b1fb8750ee2" containerName="istio-proxy" Jul 16 11:00:00.152483 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.152453 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="220f2921-5ca5-4285-ac0c-5b1fb8750ee2" containerName="istio-proxy" Jul 16 11:00:00.155395 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.155377 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" Jul 16 11:00:00.158054 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.158036 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlcx9\"" Jul 16 11:00:00.165788 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.165769 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" Jul 16 11:00:00.166695 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.166675 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq"] Jul 16 11:00:00.298358 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.298334 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq"] Jul 16 11:00:00.300598 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:00:00.300568 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedda607_6082_4716_9f05_1132283d8582.slice/crio-0eec32ee646990b0ff124317e7be9a0f1f65299467c34e169fcea004c92416ad WatchSource:0}: Error finding container 0eec32ee646990b0ff124317e7be9a0f1f65299467c34e169fcea004c92416ad: Status 404 returned error can't find the container with id 0eec32ee646990b0ff124317e7be9a0f1f65299467c34e169fcea004c92416ad Jul 16 11:00:00.826763 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:00.826723 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:01.170737 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:01.170708 2568 generic.go:356] "Generic (PLEG): container finished" podID="cedda607-6082-4716-9f05-1132283d8582" containerID="c9f4f0c8e74229fb3e394895f91778a7a7e11f35fadd321164e1d81dfb59c072" exitCode=0 Jul 16 11:00:01.171059 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:01.170750 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" event={"ID":"cedda607-6082-4716-9f05-1132283d8582","Type":"ContainerDied","Data":"c9f4f0c8e74229fb3e394895f91778a7a7e11f35fadd321164e1d81dfb59c072"} Jul 16 11:00:01.171059 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:01.170797 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" event={"ID":"cedda607-6082-4716-9f05-1132283d8582","Type":"ContainerStarted","Data":"0eec32ee646990b0ff124317e7be9a0f1f65299467c34e169fcea004c92416ad"} Jul 16 11:00:02.302872 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:02.302846 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" Jul 16 11:00:03.181005 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:03.180973 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" Jul 16 11:00:03.181005 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:03.181003 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736660-djrsq" event={"ID":"cedda607-6082-4716-9f05-1132283d8582","Type":"ContainerDied","Data":"0eec32ee646990b0ff124317e7be9a0f1f65299467c34e169fcea004c92416ad"} Jul 16 11:00:03.181222 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:03.181053 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0eec32ee646990b0ff124317e7be9a0f1f65299467c34e169fcea004c92416ad" Jul 16 11:00:07.555609 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:07.555577 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:07.655606 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:07.655575 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:08.377286 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:08.377247 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:08.497255 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:08.497221 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:08.599447 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:08.599416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:10.303767 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:10.303739 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:17.785599 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:17.785560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:17.892383 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:17.892351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:18.554068 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:18.554026 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:19.864733 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:19.864699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:24.860121 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:24.860089 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:24.959215 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:24.959167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:25.114237 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.114154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:25.474748 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.474667 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn"] Jul 16 11:00:25.475109 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.475090 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cedda607-6082-4716-9f05-1132283d8582" containerName="cleanup" Jul 16 11:00:25.475207 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.475110 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cedda607-6082-4716-9f05-1132283d8582" containerName="cleanup" Jul 16 11:00:25.475272 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.475239 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cedda607-6082-4716-9f05-1132283d8582" containerName="cleanup" Jul 16 11:00:25.478143 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.478121 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.485567 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.485548 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3b9fb924-openshift-default-dockercfg-9kshx\"" Jul 16 11:00:25.501009 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.500984 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn"] Jul 16 11:00:25.652928 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.652873 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-certs\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653106 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.652944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-data\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653106 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.652983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931d5011-da48-49c4-9ae5-d241e5416bae-istio-podinfo\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653106 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.653055 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931d5011-da48-49c4-9ae5-d241e5416bae-istiod-ca-cert\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653106 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.653091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2tf\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-kube-api-access-lx2tf\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653283 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.653142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-socket\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653283 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.653158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-envoy\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653283 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.653186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-istio-token\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.653283 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.653261 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-credential-socket\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.689678 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.689654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:25.753890 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.753812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-socket\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.753890 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.753847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-envoy\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.753890 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.753862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-istio-token\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754094 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754026 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-credential-socket\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754094 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754089 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-certs\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754176 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754132 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-data\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754228 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931d5011-da48-49c4-9ae5-d241e5416bae-istio-podinfo\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754282 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931d5011-da48-49c4-9ae5-d241e5416bae-istiod-ca-cert\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754282 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2tf\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-kube-api-access-lx2tf\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754282 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754269 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-socket\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754457 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-credential-socket\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754514 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754451 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-certs\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.754774 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.754740 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-data\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.755101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.755084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931d5011-da48-49c4-9ae5-d241e5416bae-istiod-ca-cert\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.756242 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.756222 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-envoy\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.756508 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.756488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931d5011-da48-49c4-9ae5-d241e5416bae-istio-podinfo\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.764569 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.764550 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-istio-token\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.764651 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.764570 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2tf\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-kube-api-access-lx2tf\") pod \"e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.788617 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.788596 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:25.919342 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.919305 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:25.923967 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.923942 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn"] Jul 16 11:00:25.927608 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:00:25.927582 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931d5011_da48_49c4_9ae5_d241e5416bae.slice/crio-62a4132c3f4df3578a907205437c3b5465a2c5ff553ba894a64185d364832dfa WatchSource:0}: Error finding container 62a4132c3f4df3578a907205437c3b5465a2c5ff553ba894a64185d364832dfa: Status 404 returned error can't find the container with id 62a4132c3f4df3578a907205437c3b5465a2c5ff553ba894a64185d364832dfa Jul 16 11:00:25.929613 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.929588 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:00:25.929677 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.929650 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:00:25.929719 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:25.929700 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:00:26.294333 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:26.294302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" event={"ID":"931d5011-da48-49c4-9ae5-d241e5416bae","Type":"ContainerStarted","Data":"964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a"} Jul 16 11:00:26.294502 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:26.294349 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" event={"ID":"931d5011-da48-49c4-9ae5-d241e5416bae","Type":"ContainerStarted","Data":"62a4132c3f4df3578a907205437c3b5465a2c5ff553ba894a64185d364832dfa"} Jul 16 11:00:26.319383 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:26.319340 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podStartSLOduration=1.319325738 podStartE2EDuration="1.319325738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:00:26.318741131 +0000 UTC m=+1685.455467800" watchObservedRunningTime="2026-07-16 11:00:26.319325738 +0000 UTC m=+1685.456052410" Jul 16 11:00:26.789555 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:26.789524 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:26.794204 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:26.794180 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:27.296045 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:27.296012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:27.299955 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:27.299932 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:27.300738 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:27.300718 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:27.406385 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:27.406352 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:28.116422 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:28.116385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:28.216767 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:28.216731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:32.801669 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:32.801635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:33.480495 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:33.480444 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:33.574001 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:33.573964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:33.933181 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:33.933150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:36.905208 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:36.905173 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:37.013948 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:37.013902 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:38.355198 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:38.355171 2568 generic.go:356] "Generic (PLEG): container finished" podID="931d5011-da48-49c4-9ae5-d241e5416bae" containerID="964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a" exitCode=0 Jul 16 11:00:38.355597 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:38.355251 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" event={"ID":"931d5011-da48-49c4-9ae5-d241e5416bae","Type":"ContainerDied","Data":"964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a"} Jul 16 11:00:38.355642 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:38.355616 2568 scope.go:117] "RemoveContainer" containerID="964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a" Jul 16 11:00:38.356672 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:38.356648 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:00:38.356734 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:38.356710 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:00:38.356778 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:38.356739 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:00:39.363820 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:39.363786 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" event={"ID":"931d5011-da48-49c4-9ae5-d241e5416bae","Type":"ContainerStarted","Data":"4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286"} Jul 16 11:00:39.789626 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:39.789539 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:39.790807 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:39.790782 2568 patch_prober.go:28] interesting pod/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 16 11:00:39.790945 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:39.790827 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 16 11:00:40.789694 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:40.789658 2568 patch_prober.go:28] interesting pod/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 16 11:00:40.790103 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:40.789713 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 16 11:00:41.058175 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:41.058017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:41.123992 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:41.123945 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:41.789547 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:41.789508 2568 patch_prober.go:28] interesting pod/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 16 11:00:41.789751 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:41.789568 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 16 11:00:42.300590 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:42.300556 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:42.789772 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:42.789740 2568 patch_prober.go:28] interesting pod/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 16 11:00:42.790161 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:42.789796 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 16 11:00:43.792863 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:43.792829 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:43.793722 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:43.793705 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:47.453713 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:47.453675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:47.541145 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:47.541118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:53.503446 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:53.503407 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:53.605198 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:53.605166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:53.787827 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:53.787748 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn"] Jul 16 11:00:53.788057 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:53.788030 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" containerID="cri-o://4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286" gracePeriod=30 Jul 16 11:00:53.801250 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:53.801225 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:54.040873 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:54.040785 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:54.171125 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:54.171085 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:54.266814 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:54.266780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:54.380963 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:54.380930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:00:58.795956 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:58.795917 2568 patch_prober.go:28] interesting pod/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:50430->10.132.0.41:15021: read: connection reset by peer" start-of-body= Jul 16 11:00:58.796415 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:58.795999 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:50430->10.132.0.41:15021: read: connection reset by peer" Jul 16 11:00:59.038401 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.038379 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:59.142981 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.142953 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-istio-token\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143123 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.142993 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931d5011-da48-49c4-9ae5-d241e5416bae-istio-podinfo\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143123 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143015 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-envoy\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143123 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143041 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931d5011-da48-49c4-9ae5-d241e5416bae-istiod-ca-cert\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143123 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143086 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-data\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143293 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143168 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-socket\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143293 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143196 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx2tf\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-kube-api-access-lx2tf\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143293 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143226 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-credential-socket\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143293 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143268 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-certs\") pod \"931d5011-da48-49c4-9ae5-d241e5416bae\" (UID: \"931d5011-da48-49c4-9ae5-d241e5416bae\") " Jul 16 11:00:59.143534 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143504 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/931d5011-da48-49c4-9ae5-d241e5416bae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:00:59.143761 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143733 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:00:59.143891 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143745 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:00:59.143891 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.143800 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:00:59.144235 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.144206 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-data" (OuterVolumeSpecName: "istio-data") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:00:59.145261 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.145228 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:00:59.145373 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.145270 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-istio-token" (OuterVolumeSpecName: "istio-token") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:00:59.145373 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.145321 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/931d5011-da48-49c4-9ae5-d241e5416bae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:00:59.145373 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.145320 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-kube-api-access-lx2tf" (OuterVolumeSpecName: "kube-api-access-lx2tf") pod "931d5011-da48-49c4-9ae5-d241e5416bae" (UID: "931d5011-da48-49c4-9ae5-d241e5416bae"). InnerVolumeSpecName "kube-api-access-lx2tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244017 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244044 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx2tf\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-kube-api-access-lx2tf\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244055 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244064 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244073 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931d5011-da48-49c4-9ae5-d241e5416bae-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244081 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931d5011-da48-49c4-9ae5-d241e5416bae-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244088 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244097 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931d5011-da48-49c4-9ae5-d241e5416bae-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.244101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.244104 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931d5011-da48-49c4-9ae5-d241e5416bae-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:00:59.454947 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.454890 2568 generic.go:356] "Generic (PLEG): container finished" podID="931d5011-da48-49c4-9ae5-d241e5416bae" containerID="4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286" exitCode=0 Jul 16 11:00:59.455107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.454982 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" Jul 16 11:00:59.455107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.454980 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" event={"ID":"931d5011-da48-49c4-9ae5-d241e5416bae","Type":"ContainerDied","Data":"4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286"} Jul 16 11:00:59.455107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.455036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn" event={"ID":"931d5011-da48-49c4-9ae5-d241e5416bae","Type":"ContainerDied","Data":"62a4132c3f4df3578a907205437c3b5465a2c5ff553ba894a64185d364832dfa"} Jul 16 11:00:59.455107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.455058 2568 scope.go:117] "RemoveContainer" containerID="4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286" Jul 16 11:00:59.463630 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.463612 2568 scope.go:117] "RemoveContainer" containerID="964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a" Jul 16 11:00:59.470959 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.470945 2568 scope.go:117] "RemoveContainer" containerID="4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286" Jul 16 11:00:59.471196 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:00:59.471179 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286\": container with ID starting with 4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286 not found: ID does not exist" containerID="4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286" Jul 16 11:00:59.471245 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.471204 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286"} err="failed to get container status \"4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286\": rpc error: code = NotFound desc = could not find container \"4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286\": container with ID starting with 4e4f6adb90dbc5b5dce6d192fedcfbaa4b73dc12d3dd6871dcb867cd7610a286 not found: ID does not exist" Jul 16 11:00:59.471245 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.471222 2568 scope.go:117] "RemoveContainer" containerID="964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a" Jul 16 11:00:59.471441 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:00:59.471422 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a\": container with ID starting with 964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a not found: ID does not exist" containerID="964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a" Jul 16 11:00:59.471495 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.471451 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a"} err="failed to get container status \"964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a\": rpc error: code = NotFound desc = could not find container \"964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a\": container with ID starting with 964b9da51e942bb30a46964e51c172d59bf5da2bc0c286f6bdab2aa6b495a72a not found: ID does not exist" Jul 16 11:00:59.474566 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.474544 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn"] Jul 16 11:00:59.479090 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:00:59.479067 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3b9fb924-openshift-default-76b857c6bc-5hvhn"] Jul 16 11:01:00.486824 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:00.486788 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:00.599143 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:00.599109 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:00.691728 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:00.691696 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:00.892272 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:00.892241 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:01.387657 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:01.387629 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" path="/var/lib/kubelet/pods/931d5011-da48-49c4-9ae5-d241e5416bae/volumes" Jul 16 11:01:07.281231 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:07.281164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:07.384036 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:07.384007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:15.015024 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:15.014940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:15.113066 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:15.113031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:15.213055 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:15.213028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:15.927466 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:15.927435 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:16.026790 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:16.026754 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:16.131629 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:16.131599 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:17.543187 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:17.543150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:23.814957 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:23.814897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:24.066522 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:24.066449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:24.672008 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:24.671972 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:31.028425 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:31.028393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:31.151613 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:31.151578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:31.286596 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:31.286517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:42.925115 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:42.925063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:01:43.023081 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:01:43.023049 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:03.309919 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:03.309873 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:09.558776 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:09.558738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:09.653543 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:09.653506 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:09.947980 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:09.947944 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:10.282313 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282232 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9"] Jul 16 11:02:10.282629 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282617 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" Jul 16 11:02:10.282672 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282631 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" Jul 16 11:02:10.282672 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282643 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" Jul 16 11:02:10.282672 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282649 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" Jul 16 11:02:10.282771 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282733 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" Jul 16 11:02:10.282771 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.282745 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="931d5011-da48-49c4-9ae5-d241e5416bae" containerName="istio-proxy" Jul 16 11:02:10.285864 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.285845 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.289402 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.289385 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-5e601d-openshift-default-dockercfg-7964q\"" Jul 16 11:02:10.299590 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.299570 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9"] Jul 16 11:02:10.310932 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.310892 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:10.381380 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.381353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:10.428876 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.428846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-credential-socket\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429023 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.428880 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-certs\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429023 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.428937 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-data\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429023 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.428963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-podinfo\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429023 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.429001 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-socket\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429201 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.429071 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-envoy\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429201 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.429095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istiod-ca-cert\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429201 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.429130 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpxw\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-kube-api-access-hhpxw\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.429201 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.429152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-token\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.529783 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.529749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-credential-socket\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.529783 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.529781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-certs\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530122 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.529809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-data\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530122 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.529829 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-podinfo\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530122 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.529854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-socket\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530122 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.529897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-envoy\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530318 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.530122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istiod-ca-cert\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530318 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.530205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpxw\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-kube-api-access-hhpxw\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530549 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.530221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-data\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.530786 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.530761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-token\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.531367 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.531338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-credential-socket\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.531469 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.531377 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-certs\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.531676 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.531644 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istiod-ca-cert\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.532466 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.532406 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-socket\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.538070 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.537612 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-envoy\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.538070 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.537949 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-podinfo\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.540539 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.540514 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-token\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.540669 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.540651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpxw\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-kube-api-access-hhpxw\") pod \"e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.595586 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.595567 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:10.730039 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.730014 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9"] Jul 16 11:02:10.732683 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:02:10.732655 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e1d3395_e476_4b8a_bb00_1a699b86ca7f.slice/crio-0ecba837d0c7560d0eda97daed40281e965c99b5ef5406edf8a14f91517e44e0 WatchSource:0}: Error finding container 0ecba837d0c7560d0eda97daed40281e965c99b5ef5406edf8a14f91517e44e0: Status 404 returned error can't find the container with id 0ecba837d0c7560d0eda97daed40281e965c99b5ef5406edf8a14f91517e44e0 Jul 16 11:02:10.734860 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.734829 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:02:10.734936 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.734891 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:02:10.734983 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.734933 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:02:10.783124 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:10.783100 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" event={"ID":"5e1d3395-e476-4b8a-bb00-1a699b86ca7f","Type":"ContainerStarted","Data":"0ecba837d0c7560d0eda97daed40281e965c99b5ef5406edf8a14f91517e44e0"} Jul 16 11:02:11.788420 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:11.788384 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" event={"ID":"5e1d3395-e476-4b8a-bb00-1a699b86ca7f","Type":"ContainerStarted","Data":"ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d"} Jul 16 11:02:11.814105 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:11.814058 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podStartSLOduration=1.814042573 podStartE2EDuration="1.814042573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:02:11.812362121 +0000 UTC m=+1790.949088804" watchObservedRunningTime="2026-07-16 11:02:11.814042573 +0000 UTC m=+1790.950769314" Jul 16 11:02:12.596100 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:12.596067 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:12.600776 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:12.600753 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:12.793847 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:12.793815 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:12.794681 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:12.794663 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:12.867191 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:12.867090 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:12.987852 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:12.987764 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:13.298901 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:13.298800 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:13.830855 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:13.830826 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:13.980650 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:13.980620 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:18.406892 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:18.406855 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:18.760474 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:18.760445 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:19.097234 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:19.097187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:19.257043 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:19.257012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:21.215725 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.215686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:21.353731 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.353601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:21.401380 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.401357 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:02:21.403371 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.403347 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:02:21.404176 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.404156 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 11:02:21.406241 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.406223 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 11:02:21.482537 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.482461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:21.902581 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:21.902550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:22.121023 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:22.120989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:22.229261 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:22.229177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:26.726326 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:26.726292 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:27.809592 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.809550 2568 patch_prober.go:28] interesting pod/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:36568->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 16 11:02:27.810006 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.809634 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:36568->10.132.0.42:15021: read: connection reset by peer" Jul 16 11:02:27.870762 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.870730 2568 generic.go:356] "Generic (PLEG): container finished" podID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerID="ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d" exitCode=0 Jul 16 11:02:27.870888 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.870799 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" event={"ID":"5e1d3395-e476-4b8a-bb00-1a699b86ca7f","Type":"ContainerDied","Data":"ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d"} Jul 16 11:02:27.871219 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.871198 2568 scope.go:117] "RemoveContainer" containerID="ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d" Jul 16 11:02:27.872258 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.872233 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:02:27.872334 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.872320 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:02:27.872376 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:27.872354 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:02:28.876811 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:28.876777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" event={"ID":"5e1d3395-e476-4b8a-bb00-1a699b86ca7f","Type":"ContainerStarted","Data":"045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58"} Jul 16 11:02:29.596588 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:29.596557 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:29.596770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:29.596750 2568 patch_prober.go:28] interesting pod/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 16 11:02:29.596815 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:29.596798 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 16 11:02:30.596000 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:30.595964 2568 patch_prober.go:28] interesting pod/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 16 11:02:30.596390 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:30.596021 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 16 11:02:31.596840 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:31.596807 2568 patch_prober.go:28] interesting pod/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 16 11:02:31.597221 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:31.596870 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 16 11:02:32.596809 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:32.596774 2568 patch_prober.go:28] interesting pod/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 16 11:02:32.597009 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:32.596827 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 16 11:02:33.599696 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:33.599667 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:33.600088 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:33.600042 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:33.600620 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:33.600603 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:02:35.988988 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:35.988946 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:36.088413 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:36.088378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:39.858712 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:39.858672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:43.048416 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:43.048385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:02:43.146741 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:02:43.146706 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:03:06.424699 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:03:06.424660 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:03:41.869922 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:03:41.869883 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:03:42.867077 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:03:42.867042 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:03:43.350033 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:03:43.350003 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:03:48.417554 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:03:48.417506 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:25.025446 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:25.025412 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:25.962979 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:25.962930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:32.209507 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:32.209473 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:32.304230 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:32.304182 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:32.479916 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:32.479829 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9"] Jul 16 11:04:32.480080 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:32.480058 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" containerID="cri-o://045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58" gracePeriod=30 Jul 16 11:04:32.489486 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:32.489447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:32.759837 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:32.759732 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:33.198774 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:33.198745 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:33.601950 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:33.601919 2568 patch_prober.go:28] interesting pod/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 11:04:33.602300 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:33.601990 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 11:04:37.728262 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.728238 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:04:37.784655 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784597 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-podinfo\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.784770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784662 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-data\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.784770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784686 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpxw\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-kube-api-access-hhpxw\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.784770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784724 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-certs\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.784770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784739 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istiod-ca-cert\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.784770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784754 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-envoy\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.785054 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784837 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-credential-socket\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.785054 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784874 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-token\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.785054 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.784901 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-socket\") pod \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\" (UID: \"5e1d3395-e476-4b8a-bb00-1a699b86ca7f\") " Jul 16 11:04:37.785215 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785071 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:37.785275 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785209 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:04:37.785325 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785308 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.785392 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785326 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.785490 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785464 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:37.785641 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785610 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:37.785755 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.785731 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:37.786896 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.786872 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:37.787037 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.786897 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-kube-api-access-hhpxw" (OuterVolumeSpecName: "kube-api-access-hhpxw") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "kube-api-access-hhpxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:04:37.787199 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.787173 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:04:37.787291 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.787229 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e1d3395-e476-4b8a-bb00-1a699b86ca7f" (UID: "5e1d3395-e476-4b8a-bb00-1a699b86ca7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:04:37.886484 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886456 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.886484 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886480 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhpxw\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-kube-api-access-hhpxw\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.886484 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886489 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.886677 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886498 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.886677 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886510 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.886677 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886519 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:37.886677 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:37.886526 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e1d3395-e476-4b8a-bb00-1a699b86ca7f-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:04:38.484302 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.484271 2568 generic.go:356] "Generic (PLEG): container finished" podID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerID="045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58" exitCode=0 Jul 16 11:04:38.484467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.484319 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" Jul 16 11:04:38.484467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.484333 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" event={"ID":"5e1d3395-e476-4b8a-bb00-1a699b86ca7f","Type":"ContainerDied","Data":"045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58"} Jul 16 11:04:38.484467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.484371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9" event={"ID":"5e1d3395-e476-4b8a-bb00-1a699b86ca7f","Type":"ContainerDied","Data":"0ecba837d0c7560d0eda97daed40281e965c99b5ef5406edf8a14f91517e44e0"} Jul 16 11:04:38.484467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.484392 2568 scope.go:117] "RemoveContainer" containerID="045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58" Jul 16 11:04:38.502674 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.502645 2568 scope.go:117] "RemoveContainer" containerID="ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d" Jul 16 11:04:38.511210 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.511191 2568 scope.go:117] "RemoveContainer" containerID="045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58" Jul 16 11:04:38.511450 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:04:38.511434 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58\": container with ID starting with 045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58 not found: ID does not exist" containerID="045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58" Jul 16 11:04:38.511496 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.511457 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58"} err="failed to get container status \"045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58\": rpc error: code = NotFound desc = could not find container \"045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58\": container with ID starting with 045f99121c9f214e019ff8c84def7cdfefeea7030ad664933db04ddb4e74ee58 not found: ID does not exist" Jul 16 11:04:38.511496 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.511473 2568 scope.go:117] "RemoveContainer" containerID="ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d" Jul 16 11:04:38.511677 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:04:38.511658 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d\": container with ID starting with ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d not found: ID does not exist" containerID="ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d" Jul 16 11:04:38.511715 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.511686 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d"} err="failed to get container status \"ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d\": rpc error: code = NotFound desc = could not find container \"ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d\": container with ID starting with ca127943533b19f4b738935822d24c8328ca9bd21422a559bfa6ee77e490f55d not found: ID does not exist" Jul 16 11:04:38.521707 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.521687 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9"] Jul 16 11:04:38.537168 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:38.537149 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-5e601d-openshift-default-8dcf5947f-cmbc9"] Jul 16 11:04:39.387899 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:39.387866 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" path="/var/lib/kubelet/pods/5e1d3395-e476-4b8a-bb00-1a699b86ca7f/volumes" Jul 16 11:04:40.118337 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:40.118305 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:40.299263 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:40.299230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:46.459637 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:46.459606 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:47.697180 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697147 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx"] Jul 16 11:04:47.697571 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697556 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" Jul 16 11:04:47.697624 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697573 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" Jul 16 11:04:47.697624 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697588 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" Jul 16 11:04:47.697624 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697594 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" Jul 16 11:04:47.697715 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697659 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" Jul 16 11:04:47.697715 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.697668 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e1d3395-e476-4b8a-bb00-1a699b86ca7f" containerName="istio-proxy" Jul 16 11:04:47.702041 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.702023 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.704765 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.704740 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-270141-openshift-default-dockercfg-qd6lt\"" Jul 16 11:04:47.716376 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.716351 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx"] Jul 16 11:04:47.769780 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769745 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-data\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.769780 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istiod-ca-cert\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770018 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769812 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-credential-socket\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770018 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k252f\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-kube-api-access-k252f\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770018 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-envoy\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770018 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769956 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-token\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770018 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.769984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-socket\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770018 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.770003 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-certs\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.770233 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.770082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-podinfo\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871321 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-token\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871321 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871324 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-socket\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871542 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-certs\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871542 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871378 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-podinfo\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871542 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-data\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871542 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871427 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istiod-ca-cert\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871742 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-credential-socket\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871742 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871586 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k252f\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-kube-api-access-k252f\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871742 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-envoy\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871901 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871760 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-socket\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871901 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871813 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-data\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.871901 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-certs\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.872078 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.871971 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-credential-socket\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.872170 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.872153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istiod-ca-cert\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.873779 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.873758 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-envoy\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.873863 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.873822 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-podinfo\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.883783 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.883759 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-token\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:47.884046 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:47.884031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k252f\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-kube-api-access-k252f\") pod \"e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:48.012406 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.012330 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:48.149000 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.148944 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx"] Jul 16 11:04:48.151348 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:04:48.151322 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853c1f7e_43e1_4ae1_ab29_29a4b7984ba3.slice/crio-fbd25b5e0d75c47c250910d2b96c1eadb635ae9ec483f192f686202cd2cb2472 WatchSource:0}: Error finding container fbd25b5e0d75c47c250910d2b96c1eadb635ae9ec483f192f686202cd2cb2472: Status 404 returned error can't find the container with id fbd25b5e0d75c47c250910d2b96c1eadb635ae9ec483f192f686202cd2cb2472 Jul 16 11:04:48.153493 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.153463 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:04:48.153575 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.153524 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:04:48.153575 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.153552 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:04:48.530800 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.530765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" event={"ID":"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3","Type":"ContainerStarted","Data":"85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd"} Jul 16 11:04:48.531003 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.530814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" event={"ID":"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3","Type":"ContainerStarted","Data":"fbd25b5e0d75c47c250910d2b96c1eadb635ae9ec483f192f686202cd2cb2472"} Jul 16 11:04:48.553652 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.553599 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podStartSLOduration=1.553580127 podStartE2EDuration="1.553580127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:48.551588906 +0000 UTC m=+1947.688315578" watchObservedRunningTime="2026-07-16 11:04:48.553580127 +0000 UTC m=+1947.690306799" Jul 16 11:04:48.861983 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:48.861895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:49.013275 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:49.013251 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:49.017971 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:49.017949 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:49.534026 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:49.533990 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:49.536104 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:49.536082 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:49.536873 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:49.536855 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:04:50.153202 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:50.153172 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:50.248821 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:50.248789 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:50.419262 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:50.419177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:50.519067 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:50.519037 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:55.360192 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:55.360152 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:55.786839 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:55.786755 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:55.935592 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:55.935558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:56.090633 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:56.090593 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:58.125797 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:58.125722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:58.226682 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:58.226650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:58.327061 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:58.327025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:58.589088 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:58.589053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:59.210121 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:59.210086 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:04:59.554259 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:04:59.554169 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:01.601178 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:01.601142 2568 generic.go:356] "Generic (PLEG): container finished" podID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerID="85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd" exitCode=0 Jul 16 11:05:01.601706 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:01.601217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" event={"ID":"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3","Type":"ContainerDied","Data":"85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd"} Jul 16 11:05:01.601783 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:01.601737 2568 scope.go:117] "RemoveContainer" containerID="85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd" Jul 16 11:05:01.602722 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:01.602698 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:01.602779 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:01.602759 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:01.602815 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:01.602787 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:02.608102 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:02.608069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" event={"ID":"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3","Type":"ContainerStarted","Data":"91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2"} Jul 16 11:05:03.012762 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:03.012668 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:05:03.013809 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:03.013784 2568 patch_prober.go:28] interesting pod/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 16 11:05:03.013949 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:03.013831 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 16 11:05:03.637489 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:03.637448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:04.013699 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:04.013600 2568 patch_prober.go:28] interesting pod/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 16 11:05:04.013699 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:04.013667 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 16 11:05:04.536204 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:04.536160 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:05:05.012794 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:05.012760 2568 patch_prober.go:28] interesting pod/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 16 11:05:05.013227 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:05.012829 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 16 11:05:06.013645 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:06.013612 2568 patch_prober.go:28] interesting pod/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 16 11:05:06.014045 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:06.013661 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 16 11:05:07.012978 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:07.012944 2568 patch_prober.go:28] interesting pod/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 16 11:05:07.013157 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:07.012997 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 16 11:05:08.017446 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:08.017416 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:05:08.018414 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:08.018393 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:05:11.682897 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:11.682861 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:11.803394 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:11.803357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:12.407676 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:12.407644 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:17.127585 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:17.127505 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:17.219278 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:17.219242 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:17.761829 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:17.761794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:19.573598 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:19.573563 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:20.065589 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:20.065506 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:20.197380 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:20.197345 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:26.060753 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:26.060717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:26.165235 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:26.165201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:26.411865 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:26.411829 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx"] Jul 16 11:05:26.412080 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:26.412053 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" containerID="cri-o://91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2" gracePeriod=30 Jul 16 11:05:26.443313 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:26.443271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:26.670871 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:26.670789 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:27.168016 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:27.167979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:31.665514 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.665490 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:05:31.745631 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.745552 2568 generic.go:356] "Generic (PLEG): container finished" podID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerID="91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2" exitCode=0 Jul 16 11:05:31.745631 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.745604 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" Jul 16 11:05:31.745788 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.745640 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" event={"ID":"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3","Type":"ContainerDied","Data":"91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2"} Jul 16 11:05:31.745788 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.745695 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx" event={"ID":"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3","Type":"ContainerDied","Data":"fbd25b5e0d75c47c250910d2b96c1eadb635ae9ec483f192f686202cd2cb2472"} Jul 16 11:05:31.745788 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.745718 2568 scope.go:117] "RemoveContainer" containerID="91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2" Jul 16 11:05:31.756189 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.756174 2568 scope.go:117] "RemoveContainer" containerID="85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd" Jul 16 11:05:31.763817 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.763801 2568 scope.go:117] "RemoveContainer" containerID="91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2" Jul 16 11:05:31.764800 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:05:31.764779 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2\": container with ID starting with 91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2 not found: ID does not exist" containerID="91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2" Jul 16 11:05:31.764897 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.764809 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2"} err="failed to get container status \"91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2\": rpc error: code = NotFound desc = could not find container \"91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2\": container with ID starting with 91436d3782b6ecc11af87d3fbcce52ab72cb8e8310288c3ffe0ded13c3a44cf2 not found: ID does not exist" Jul 16 11:05:31.764897 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.764826 2568 scope.go:117] "RemoveContainer" containerID="85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd" Jul 16 11:05:31.765075 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:05:31.765057 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd\": container with ID starting with 85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd not found: ID does not exist" containerID="85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd" Jul 16 11:05:31.765126 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.765080 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd"} err="failed to get container status \"85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd\": rpc error: code = NotFound desc = could not find container \"85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd\": container with ID starting with 85c27c20fb8fa052315bb57396a3fe9dc75ac4bde70a89fd4bce4d44f03389bd not found: ID does not exist" Jul 16 11:05:31.773160 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773140 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istiod-ca-cert\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773260 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773180 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-credential-socket\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773260 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773197 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-socket\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773260 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773220 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-certs\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773260 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773248 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-data\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773493 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773284 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-podinfo\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773493 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773318 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-envoy\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773493 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773383 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k252f\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-kube-api-access-k252f\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773493 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773404 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-token\") pod \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\" (UID: \"853c1f7e-43e1-4ae1-ab29-29a4b7984ba3\") " Jul 16 11:05:31.773758 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773507 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:31.773758 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773611 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:31.773758 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773516 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:05:31.773758 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773703 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:31.773994 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773959 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.773994 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773981 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.774103 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.773994 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.774103 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.774007 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.774613 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.774572 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-data" (OuterVolumeSpecName: "istio-data") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:31.775683 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.775659 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:05:31.776016 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.775992 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:31.776100 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.776073 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-token" (OuterVolumeSpecName: "istio-token") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:05:31.776172 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.776125 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-kube-api-access-k252f" (OuterVolumeSpecName: "kube-api-access-k252f") pod "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" (UID: "853c1f7e-43e1-4ae1-ab29-29a4b7984ba3"). InnerVolumeSpecName "kube-api-access-k252f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:05:31.874968 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.874945 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.874968 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.874966 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.875101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.874974 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.875101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.874984 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k252f\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-kube-api-access-k252f\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:31.875101 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:31.874993 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:05:32.069720 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:32.069686 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx"] Jul 16 11:05:32.073709 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:32.073685 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-270141-openshift-default-55d889cf8f-m4bdx"] Jul 16 11:05:32.493236 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:32.493200 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:33.388773 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:33.388744 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" path="/var/lib/kubelet/pods/853c1f7e-43e1-4ae1-ab29-29a4b7984ba3/volumes" Jul 16 11:05:33.490945 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:33.490898 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:38.060543 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:38.060510 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:38.135484 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:38.135451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:38.716425 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:38.716391 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:38.815335 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:38.815300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:40.156787 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:40.156760 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:40.243335 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:40.243302 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:40.338229 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:40.338198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:40.571947 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:40.571898 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:40.918993 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:40.918886 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:41.233245 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.233167 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7"] Jul 16 11:05:41.233697 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.233678 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" Jul 16 11:05:41.233773 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.233701 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" Jul 16 11:05:41.233832 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.233823 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" Jul 16 11:05:41.238307 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.238285 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.241763 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.241743 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-046938-openshift-default-dockercfg-w8cjj\"" Jul 16 11:05:41.253641 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.253622 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7"] Jul 16 11:05:41.306224 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.306199 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:41.356029 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356004 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-data\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356179 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istiod-ca-cert\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356179 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356076 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-credential-socket\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356179 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356093 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-certs\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356179 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356111 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-token\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356179 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356131 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69wkr\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-kube-api-access-69wkr\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356179 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-socket\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356407 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-envoy\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.356407 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.356220 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-podinfo\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457422 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-envoy\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457588 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457436 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-podinfo\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457588 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457489 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-data\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457707 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457683 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istiod-ca-cert\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457768 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-credential-socket\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457768 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-certs\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457874 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-token\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.457874 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457829 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69wkr\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-kube-api-access-69wkr\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.458012 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457889 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-data\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.458012 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.457944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-socket\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.458128 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.458093 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-credential-socket\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.458263 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.458239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-socket\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.458461 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.458438 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-certs\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.458515 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.458499 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istiod-ca-cert\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.459773 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.459753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-envoy\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.460011 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.459995 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-podinfo\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.473003 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.472976 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-token\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.473117 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.473041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69wkr\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-kube-api-access-69wkr\") pod \"e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.548875 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.548806 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:41.691838 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.691809 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7"] Jul 16 11:05:41.693293 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:05:41.693266 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52a4e8c_b848_4bd9_ad9d_6669667f12aa.slice/crio-5e67a4eed78d68fc3f5fb11146cd825ae08d6380a41514a67957a03a2575e2f9 WatchSource:0}: Error finding container 5e67a4eed78d68fc3f5fb11146cd825ae08d6380a41514a67957a03a2575e2f9: Status 404 returned error can't find the container with id 5e67a4eed78d68fc3f5fb11146cd825ae08d6380a41514a67957a03a2575e2f9 Jul 16 11:05:41.695441 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.695412 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:41.695514 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.695470 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:41.695514 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.695496 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:41.720058 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.720034 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:41.803047 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.802950 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" event={"ID":"f52a4e8c-b848-4bd9-ad9d-6669667f12aa","Type":"ContainerStarted","Data":"7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9"} Jul 16 11:05:41.803047 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.803036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" event={"ID":"f52a4e8c-b848-4bd9-ad9d-6669667f12aa","Type":"ContainerStarted","Data":"5e67a4eed78d68fc3f5fb11146cd825ae08d6380a41514a67957a03a2575e2f9"} Jul 16 11:05:41.835248 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:41.835197 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" podStartSLOduration=0.835183442 podStartE2EDuration="835.183442ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:41.832884618 +0000 UTC m=+2000.969611328" watchObservedRunningTime="2026-07-16 11:05:41.835183442 +0000 UTC m=+2000.971910113" Jul 16 11:05:42.549373 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:42.549334 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:42.554432 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:42.554407 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:42.811709 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:42.811634 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:42.812565 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:42.812546 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:42.911581 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:42.911548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:43.038608 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:43.038579 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:43.145972 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:43.145942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:43.244826 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:43.244794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:43.996207 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:43.996172 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:44.241202 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:44.241167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:48.380367 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:48.380332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:48.872670 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:48.872630 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:49.179978 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:49.179874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:49.342062 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:49.341993 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:51.396620 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:51.396588 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:51.509037 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:51.508997 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:51.623020 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:51.622987 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:51.844732 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:51.844699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:52.289874 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:52.289793 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:52.391780 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:52.391730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:53.877432 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:53.877407 2568 generic.go:356] "Generic (PLEG): container finished" podID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerID="7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9" exitCode=0 Jul 16 11:05:53.877788 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:53.877446 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" event={"ID":"f52a4e8c-b848-4bd9-ad9d-6669667f12aa","Type":"ContainerDied","Data":"7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9"} Jul 16 11:05:53.877833 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:53.877795 2568 scope.go:117] "RemoveContainer" containerID="7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9" Jul 16 11:05:53.878732 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:53.878706 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:53.878830 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:53.878801 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:53.878891 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:53.878850 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 11:05:54.884337 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:54.884303 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" event={"ID":"f52a4e8c-b848-4bd9-ad9d-6669667f12aa","Type":"ContainerStarted","Data":"1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699"} Jul 16 11:05:55.550976 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:55.550934 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:55.551183 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:55.551120 2568 patch_prober.go:28] interesting pod/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 16 11:05:55.551255 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:55.551182 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 16 11:05:56.550202 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:56.550159 2568 patch_prober.go:28] interesting pod/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 16 11:05:56.550730 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:56.550236 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 16 11:05:56.891990 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:56.891949 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:05:57.549613 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:57.549577 2568 patch_prober.go:28] interesting pod/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 16 11:05:57.549926 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:57.549868 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 16 11:05:57.812069 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:57.811983 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:58.549735 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:58.549693 2568 patch_prober.go:28] interesting pod/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 16 11:05:58.549955 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:58.549760 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 16 11:05:59.552711 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:59.552683 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:05:59.553671 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:05:59.553651 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:06:04.968965 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:06:04.968926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:06:05.070587 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:06:05.070556 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:06:08.553971 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:06:08.553926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:06:13.098841 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:06:13.098808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:06:13.202829 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:06:13.202798 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:06:38.393984 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:06:38.393947 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:07:21.433017 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:07:21.432988 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:07:21.433712 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:07:21.433694 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:07:21.435859 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:07:21.435843 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 11:07:21.436300 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:07:21.436284 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 11:08:47.504503 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:47.504469 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:08:48.387404 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:48.387370 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:08:53.525743 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.525707 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5"] Jul 16 11:08:53.526328 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.526231 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" Jul 16 11:08:53.526328 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.526250 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" Jul 16 11:08:53.526450 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.526384 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="853c1f7e-43e1-4ae1-ab29-29a4b7984ba3" containerName="istio-proxy" Jul 16 11:08:53.529614 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.529593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.536660 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.536637 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"default-dockercfg-9rswc\"" Jul 16 11:08:53.536759 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.536663 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"openshift-service-ca.crt\"" Jul 16 11:08:53.536759 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.536708 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"kube-root-ca.crt\"" Jul 16 11:08:53.536759 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.536717 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"sub-test-model-046938-kserve-self-signed-certs\"" Jul 16 11:08:53.543159 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.543141 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5"] Jul 16 11:08:53.590716 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.590687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-dshm\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.590843 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.590727 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-tmp-dir\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.590843 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.590755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4slq\" (UniqueName: \"kubernetes.io/projected/9193b181-081f-4a8c-96bd-1d4db33d7371-kube-api-access-n4slq\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.590928 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.590835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-model-cache\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.590928 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.590867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9193b181-081f-4a8c-96bd-1d4db33d7371-tls-certs\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.590928 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.590885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-home\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.629150 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.629125 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:08:53.691950 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.691925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-model-cache\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692062 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.691954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9193b181-081f-4a8c-96bd-1d4db33d7371-tls-certs\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692062 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.691973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-home\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692062 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.692033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-dshm\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692062 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.692049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-tmp-dir\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692275 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.692066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4slq\" (UniqueName: \"kubernetes.io/projected/9193b181-081f-4a8c-96bd-1d4db33d7371-kube-api-access-n4slq\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692333 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.692314 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-model-cache\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692466 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.692439 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-tmp-dir\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.692538 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.692511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-home\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.694237 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.694215 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-dshm\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.694511 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.694492 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9193b181-081f-4a8c-96bd-1d4db33d7371-tls-certs\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.703515 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.703496 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4slq\" (UniqueName: \"kubernetes.io/projected/9193b181-081f-4a8c-96bd-1d4db33d7371-kube-api-access-n4slq\") pod \"sub-test-model-046938-kserve-5f47586d59-mpdg5\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.838613 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.838586 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:08:53.973441 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:53.973416 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5"] Jul 16 11:08:53.975986 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:08:53.975958 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9193b181_081f_4a8c_96bd_1d4db33d7371.slice/crio-30d9fd92f73d0a0f6030f71acbb3067840b14be4cf16ac451c5493f649b99bd5 WatchSource:0}: Error finding container 30d9fd92f73d0a0f6030f71acbb3067840b14be4cf16ac451c5493f649b99bd5: Status 404 returned error can't find the container with id 30d9fd92f73d0a0f6030f71acbb3067840b14be4cf16ac451c5493f649b99bd5 Jul 16 11:08:54.720768 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:54.720735 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" event={"ID":"9193b181-081f-4a8c-96bd-1d4db33d7371","Type":"ContainerStarted","Data":"e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c"} Jul 16 11:08:54.721129 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:54.720779 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" event={"ID":"9193b181-081f-4a8c-96bd-1d4db33d7371","Type":"ContainerStarted","Data":"30d9fd92f73d0a0f6030f71acbb3067840b14be4cf16ac451c5493f649b99bd5"} Jul 16 11:08:54.743048 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:08:54.742993 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" podStartSLOduration=1.7429801660000002 podStartE2EDuration="1.742980166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:08:54.741499892 +0000 UTC m=+2193.878226565" watchObservedRunningTime="2026-07-16 11:08:54.742980166 +0000 UTC m=+2193.879706882" Jul 16 11:09:03.839498 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:03.839458 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:09:03.839889 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:03.839513 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:09:03.851791 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:03.851765 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:09:04.777259 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:04.777226 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:09:24.154426 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:24.154378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:09:29.011626 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:29.011580 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:09:48.785871 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:09:48.785764 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:10:38.967973 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:38.967934 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:10:39.478374 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:39.478342 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5"] Jul 16 11:10:42.100938 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:42.100481 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:10:54.498043 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.497986 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" podUID="9193b181-081f-4a8c-96bd-1d4db33d7371" containerName="main" containerID="cri-o://e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c" gracePeriod=45 Jul 16 11:10:54.740649 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.740621 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:10:54.744176 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.744154 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:10:54.858175 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858148 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9193b181-081f-4a8c-96bd-1d4db33d7371-tls-certs\") pod \"9193b181-081f-4a8c-96bd-1d4db33d7371\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " Jul 16 11:10:54.858338 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858186 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4slq\" (UniqueName: \"kubernetes.io/projected/9193b181-081f-4a8c-96bd-1d4db33d7371-kube-api-access-n4slq\") pod \"9193b181-081f-4a8c-96bd-1d4db33d7371\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " Jul 16 11:10:54.858338 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858207 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-model-cache\") pod \"9193b181-081f-4a8c-96bd-1d4db33d7371\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " Jul 16 11:10:54.858338 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858238 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-tmp-dir\") pod \"9193b181-081f-4a8c-96bd-1d4db33d7371\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " Jul 16 11:10:54.858338 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858322 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-dshm\") pod \"9193b181-081f-4a8c-96bd-1d4db33d7371\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " Jul 16 11:10:54.858338 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858339 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-home\") pod \"9193b181-081f-4a8c-96bd-1d4db33d7371\" (UID: \"9193b181-081f-4a8c-96bd-1d4db33d7371\") " Jul 16 11:10:54.858603 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858506 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-model-cache" (OuterVolumeSpecName: "model-cache") pod "9193b181-081f-4a8c-96bd-1d4db33d7371" (UID: "9193b181-081f-4a8c-96bd-1d4db33d7371"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:10:54.858603 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858586 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9193b181-081f-4a8c-96bd-1d4db33d7371" (UID: "9193b181-081f-4a8c-96bd-1d4db33d7371"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:10:54.858728 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.858704 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-home" (OuterVolumeSpecName: "home") pod "9193b181-081f-4a8c-96bd-1d4db33d7371" (UID: "9193b181-081f-4a8c-96bd-1d4db33d7371"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:10:54.860464 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.860441 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-dshm" (OuterVolumeSpecName: "dshm") pod "9193b181-081f-4a8c-96bd-1d4db33d7371" (UID: "9193b181-081f-4a8c-96bd-1d4db33d7371"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:10:54.860770 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.860751 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9193b181-081f-4a8c-96bd-1d4db33d7371-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9193b181-081f-4a8c-96bd-1d4db33d7371" (UID: "9193b181-081f-4a8c-96bd-1d4db33d7371"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 11:10:54.860845 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.860774 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9193b181-081f-4a8c-96bd-1d4db33d7371-kube-api-access-n4slq" (OuterVolumeSpecName: "kube-api-access-n4slq") pod "9193b181-081f-4a8c-96bd-1d4db33d7371" (UID: "9193b181-081f-4a8c-96bd-1d4db33d7371"). InnerVolumeSpecName "kube-api-access-n4slq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:10:54.959947 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.959895 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-dshm\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:10:54.959947 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.959944 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-home\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:10:54.960142 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.959956 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9193b181-081f-4a8c-96bd-1d4db33d7371-tls-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:10:54.960142 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.959974 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4slq\" (UniqueName: \"kubernetes.io/projected/9193b181-081f-4a8c-96bd-1d4db33d7371-kube-api-access-n4slq\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:10:54.960142 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.959986 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-model-cache\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:10:54.960142 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:54.959996 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9193b181-081f-4a8c-96bd-1d4db33d7371-tmp-dir\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:10:55.280793 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.280710 2568 generic.go:356] "Generic (PLEG): container finished" podID="9193b181-081f-4a8c-96bd-1d4db33d7371" containerID="e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c" exitCode=0 Jul 16 11:10:55.280793 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.280747 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" event={"ID":"9193b181-081f-4a8c-96bd-1d4db33d7371","Type":"ContainerDied","Data":"e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c"} Jul 16 11:10:55.280793 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.280753 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" Jul 16 11:10:55.280793 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.280776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5" event={"ID":"9193b181-081f-4a8c-96bd-1d4db33d7371","Type":"ContainerDied","Data":"30d9fd92f73d0a0f6030f71acbb3067840b14be4cf16ac451c5493f649b99bd5"} Jul 16 11:10:55.280793 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.280793 2568 scope.go:117] "RemoveContainer" containerID="e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c" Jul 16 11:10:55.290157 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.290140 2568 scope.go:117] "RemoveContainer" containerID="e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c" Jul 16 11:10:55.290431 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:10:55.290415 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c\": container with ID starting with e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c not found: ID does not exist" containerID="e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c" Jul 16 11:10:55.290482 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.290441 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c"} err="failed to get container status \"e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c\": rpc error: code = NotFound desc = could not find container \"e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c\": container with ID starting with e82f29a39196b535713bbc6f65a075a5ea570b36fc167d23d1e9ae76fc06f30c not found: ID does not exist" Jul 16 11:10:55.332060 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.332035 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5"] Jul 16 11:10:55.342799 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.342778 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-046938/sub-test-model-046938-kserve-5f47586d59-mpdg5"] Jul 16 11:10:55.387285 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:10:55.387258 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9193b181-081f-4a8c-96bd-1d4db33d7371" path="/var/lib/kubelet/pods/9193b181-081f-4a8c-96bd-1d4db33d7371/volumes" Jul 16 11:11:06.064655 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:11:06.064614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:11:06.188848 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:11:06.188815 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:11:14.317417 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:11:14.317276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:11:14.442150 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:11:14.442113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:11:27.560326 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:11:27.560289 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:11:48.938753 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:11:48.938721 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:17.554153 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:17.554119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:18.472996 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:18.472967 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:21.469358 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:21.469326 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:12:21.470280 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:21.470259 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:12:21.472190 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:21.472168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 11:12:21.472900 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:21.472877 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p6m4b_918635d9-8500-4e39-94ad-25d15af7bb25/ovn-acl-logging/0.log" Jul 16 11:12:38.541851 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:38.541821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:43.940975 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:43.940871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:45.199476 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:45.199438 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:51.580327 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:51.580292 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7"] Jul 16 11:12:51.580810 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:51.580555 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" containerID="cri-o://1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699" gracePeriod=30 Jul 16 11:12:51.765597 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:51.765557 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:51.865950 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:51.865816 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:51.965255 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:51.965208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:52.357650 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:52.357605 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:56.837583 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.837560 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:12:56.857680 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.857622 2568 generic.go:356] "Generic (PLEG): container finished" podID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerID="1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699" exitCode=0 Jul 16 11:12:56.857680 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.857659 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" Jul 16 11:12:56.857680 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.857663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" event={"ID":"f52a4e8c-b848-4bd9-ad9d-6669667f12aa","Type":"ContainerDied","Data":"1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699"} Jul 16 11:12:56.857850 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.857715 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7" event={"ID":"f52a4e8c-b848-4bd9-ad9d-6669667f12aa","Type":"ContainerDied","Data":"5e67a4eed78d68fc3f5fb11146cd825ae08d6380a41514a67957a03a2575e2f9"} Jul 16 11:12:56.857850 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.857737 2568 scope.go:117] "RemoveContainer" containerID="1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699" Jul 16 11:12:56.867298 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.867281 2568 scope.go:117] "RemoveContainer" containerID="7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9" Jul 16 11:12:56.874992 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.874973 2568 scope.go:117] "RemoveContainer" containerID="1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699" Jul 16 11:12:56.875258 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:12:56.875239 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699\": container with ID starting with 1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699 not found: ID does not exist" containerID="1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699" Jul 16 11:12:56.875315 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.875269 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699"} err="failed to get container status \"1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699\": rpc error: code = NotFound desc = could not find container \"1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699\": container with ID starting with 1b15aaba330e4be6f85bbf5759baa69c229eec5cc9e0494c9b041386915ae699 not found: ID does not exist" Jul 16 11:12:56.875315 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.875285 2568 scope.go:117] "RemoveContainer" containerID="7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9" Jul 16 11:12:56.875493 ip-10-0-138-186 kubenswrapper[2568]: E0716 11:12:56.875479 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9\": container with ID starting with 7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9 not found: ID does not exist" containerID="7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9" Jul 16 11:12:56.875548 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:56.875497 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9"} err="failed to get container status \"7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9\": rpc error: code = NotFound desc = could not find container \"7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9\": container with ID starting with 7a14c4720f6ddfb02aafba075c82596fe13e75e6acf077346c9dfbc13124ffd9 not found: ID does not exist" Jul 16 11:12:57.002223 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002196 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-podinfo\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002381 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002231 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-data\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002381 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002251 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-token\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002381 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002268 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-socket\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002381 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002291 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-certs\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002604 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002402 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-credential-socket\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002604 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002436 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istiod-ca-cert\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002604 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002459 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69wkr\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-kube-api-access-69wkr\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002604 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002490 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-envoy\") pod \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\" (UID: \"f52a4e8c-b848-4bd9-ad9d-6669667f12aa\") " Jul 16 11:12:57.002604 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002549 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:12:57.002843 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002679 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:12:57.002843 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002772 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:12:57.002981 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002839 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-credential-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.002981 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.002860 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-socket\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.003099 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.003066 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:12:57.003189 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.003168 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:12:57.004334 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.004313 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:12:57.004657 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.004634 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-kube-api-access-69wkr" (OuterVolumeSpecName: "kube-api-access-69wkr") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "kube-api-access-69wkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:57.004953 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.004935 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:57.004953 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.004940 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f52a4e8c-b848-4bd9-ad9d-6669667f12aa" (UID: "f52a4e8c-b848-4bd9-ad9d-6669667f12aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:12:57.104107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104079 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istiod-ca-cert\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.104107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104101 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69wkr\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-kube-api-access-69wkr\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.104107 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104112 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-envoy\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.104316 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104121 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-podinfo\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.104316 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104129 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-data\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.104316 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104137 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-istio-token\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.104316 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.104145 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52a4e8c-b848-4bd9-ad9d-6669667f12aa-workload-certs\") on node \"ip-10-0-138-186.ec2.internal\" DevicePath \"\"" Jul 16 11:12:57.184453 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.184425 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7"] Jul 16 11:12:57.189647 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.189623 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-046938-openshift-default-69479b7c7f-cj6w7"] Jul 16 11:12:57.387940 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.387887 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" path="/var/lib/kubelet/pods/f52a4e8c-b848-4bd9-ad9d-6669667f12aa/volumes" Jul 16 11:12:57.774095 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:57.774017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:12:59.188173 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:12:59.188144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:06.036203 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:06.036170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:06.142063 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:06.142034 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:06.371945 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:06.371897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:06.623029 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:06.622941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:13.271246 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:13.271218 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-cmdh2_574cbc13-8e5e-47d1-a0ab-c4ca563a5a95/copy-manifests/0.log" Jul 16 11:13:13.281715 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:13.281695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-cmdh2_574cbc13-8e5e-47d1-a0ab-c4ca563a5a95/manager/0.log" Jul 16 11:13:14.053572 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:14.053539 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736645-rbj4h_6f18848e-74e0-4488-8d43-8e7a9c866fc6/cleanup/0.log" Jul 16 11:13:14.166623 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:14.166597 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736660-djrsq_cedda607-6082-4716-9f05-1132283d8582/cleanup/0.log" Jul 16 11:13:14.278457 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:14.278426 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-p5m8g_3fe50308-0e8d-4430-8ad4-41381e6f8837/postgres/0.log" Jul 16 11:13:14.540186 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:14.540138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:16.883416 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:16.883385 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mztsf_f3ff2214-1499-405a-af0d-f397817cc5d9/istio-proxy/0.log" Jul 16 11:13:17.015608 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:17.015581 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6nsj5_19b1da1f-e048-4802-8b67-7587c279e3ff/discovery/0.log" Jul 16 11:13:17.781306 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:17.781279 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f8555469b-j5x5d_186d512c-ec3f-4afd-b5fe-27037435710e/router/0.log" Jul 16 11:13:18.253863 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:18.253832 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-g9sjh_9574302d-4c3b-4677-b0a0-ffc6fead13a1/main/0.log" Jul 16 11:13:18.602847 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:18.602817 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-fdv7h_73a566f2-ff23-40fb-bf4a-9b3e3cc63c59/main/0.log" Jul 16 11:13:24.030117 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:24.030081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:24.112724 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:24.112695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" limit=1024 Jul 16 11:13:27.096472 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:27.096442 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hnfmv_7db05134-ea74-4a24-9dd4-47237ab1719c/global-pull-secret-syncer/0.log" Jul 16 11:13:27.260014 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:27.259983 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hkrc4_67b7facf-2c47-468e-981f-7fe93391c68f/konnectivity-agent/0.log" Jul 16 11:13:27.397452 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:27.397373 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-186.ec2.internal_3a99e495f424a8870814d989f4c4e9e7/haproxy/0.log" Jul 16 11:13:35.073518 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.073489 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-448ln_03527b67-f070-4f64-8ed1-51213054d473/cluster-monitoring-operator/0.log" Jul 16 11:13:35.122573 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.122550 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-swz8h_9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89/kube-state-metrics/0.log" Jul 16 11:13:35.149715 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.149692 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-swz8h_9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89/kube-rbac-proxy-main/0.log" Jul 16 11:13:35.181437 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.181417 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-swz8h_9ebc82f0-4a7a-4298-95d7-f9c5b20e3c89/kube-rbac-proxy-self/0.log" Jul 16 11:13:35.252094 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.252070 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-tnh58_72deebca-79f4-4364-a01e-7c29d7bf801b/monitoring-plugin/0.log" Jul 16 11:13:35.415692 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.415625 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5zmbw_d096c942-cd20-44f6-9dc8-a358aa13d0f2/node-exporter/0.log" Jul 16 11:13:35.452943 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.452900 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5zmbw_d096c942-cd20-44f6-9dc8-a358aa13d0f2/kube-rbac-proxy/0.log" Jul 16 11:13:35.489317 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.489300 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5zmbw_d096c942-cd20-44f6-9dc8-a358aa13d0f2/init-textfile/0.log" Jul 16 11:13:35.601658 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.601636 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lpmpc_013f9ee1-0515-4a70-9e02-e084ce2556b3/kube-rbac-proxy-main/0.log" Jul 16 11:13:35.638329 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.638301 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lpmpc_013f9ee1-0515-4a70-9e02-e084ce2556b3/kube-rbac-proxy-self/0.log" Jul 16 11:13:35.671635 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.671555 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lpmpc_013f9ee1-0515-4a70-9e02-e084ce2556b3/openshift-state-metrics/0.log" Jul 16 11:13:35.986703 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:35.986624 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84f56f6cff-24rgq_61b34bf5-b4fc-4d86-bda4-7c87d0b21a16/telemeter-client/0.log" Jul 16 11:13:36.014096 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:36.014072 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84f56f6cff-24rgq_61b34bf5-b4fc-4d86-bda4-7c87d0b21a16/reload/0.log" Jul 16 11:13:36.042814 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:36.042796 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84f56f6cff-24rgq_61b34bf5-b4fc-4d86-bda4-7c87d0b21a16/kube-rbac-proxy/0.log" Jul 16 11:13:37.216962 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.216933 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67"] Jul 16 11:13:37.217321 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217266 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" Jul 16 11:13:37.217321 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217277 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" Jul 16 11:13:37.217321 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217288 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9193b181-081f-4a8c-96bd-1d4db33d7371" containerName="main" Jul 16 11:13:37.217321 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217294 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9193b181-081f-4a8c-96bd-1d4db33d7371" containerName="main" Jul 16 11:13:37.217467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217367 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9193b181-081f-4a8c-96bd-1d4db33d7371" containerName="main" Jul 16 11:13:37.217467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217378 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" Jul 16 11:13:37.217467 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.217384 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52a4e8c-b848-4bd9-ad9d-6669667f12aa" containerName="istio-proxy" Jul 16 11:13:37.220394 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.220376 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.223754 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.223734 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2z85x\"/\"default-dockercfg-xjzjv\"" Jul 16 11:13:37.223868 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.223771 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z85x\"/\"openshift-service-ca.crt\"" Jul 16 11:13:37.224766 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.224747 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z85x\"/\"kube-root-ca.crt\"" Jul 16 11:13:37.237371 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.237342 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67"] Jul 16 11:13:37.336197 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.336170 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sttrn\" (UniqueName: \"kubernetes.io/projected/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-kube-api-access-sttrn\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.336311 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.336209 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-sys\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.336311 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.336227 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-proc\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.336311 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.336247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-lib-modules\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.336415 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.336366 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-podres\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437262 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-sys\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437262 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-proc\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437418 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-lib-modules\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437418 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437359 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-sys\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437418 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437369 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-proc\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437418 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-podres\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437549 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437417 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sttrn\" (UniqueName: \"kubernetes.io/projected/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-kube-api-access-sttrn\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437549 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437477 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-lib-modules\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.437549 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.437529 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-podres\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.446579 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.446550 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sttrn\" (UniqueName: \"kubernetes.io/projected/0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166-kube-api-access-sttrn\") pod \"perf-node-gather-daemonset-8cb67\" (UID: \"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.530705 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.530628 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:37.661465 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:37.661442 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67"] Jul 16 11:13:37.663489 ip-10-0-138-186 kubenswrapper[2568]: W0716 11:13:37.663465 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0d8bcf67_a2f0_4344_9fd4_cd3d98fa2166.slice/crio-61b93ee8ef82c0dbee667df025084ad4793f92bf1167e65d6deb93dc165c6ce3 WatchSource:0}: Error finding container 61b93ee8ef82c0dbee667df025084ad4793f92bf1167e65d6deb93dc165c6ce3: Status 404 returned error can't find the container with id 61b93ee8ef82c0dbee667df025084ad4793f92bf1167e65d6deb93dc165c6ce3 Jul 16 11:13:38.051989 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.051957 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/2.log" Jul 16 11:13:38.053084 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.053059 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" event={"ID":"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166","Type":"ContainerStarted","Data":"a2e2c70affa744713cb8057faa255c5031af57bed3114976050e43d439c348aa"} Jul 16 11:13:38.053223 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.053103 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" event={"ID":"0d8bcf67-a2f0-4344-9fd4-cd3d98fa2166","Type":"ContainerStarted","Data":"61b93ee8ef82c0dbee667df025084ad4793f92bf1167e65d6deb93dc165c6ce3"} Jul 16 11:13:38.053282 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.053249 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:38.056569 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.056552 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ttkps_3738f020-bb71-4cee-a084-f6a45ec10d00/console-operator/3.log" Jul 16 11:13:38.074223 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.074180 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" podStartSLOduration=1.074166806 podStartE2EDuration="1.074166806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:38.071584861 +0000 UTC m=+2477.208311546" watchObservedRunningTime="2026-07-16 11:13:38.074166806 +0000 UTC m=+2477.210893478" Jul 16 11:13:38.654558 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:38.654529 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-l74t8_345f3f95-6ca8-456a-bc44-12c43cef661a/download-server/0.log" Jul 16 11:13:39.286245 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:39.286219 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-dmswz_46ff2d4c-d1a9-478e-ac25-5fc4470890d0/volume-data-source-validator/0.log" Jul 16 11:13:40.335012 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:40.334984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2fh28_0d333f8d-b877-4374-90a4-f78d1e97a74f/dns/0.log" Jul 16 11:13:40.358076 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:40.358048 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2fh28_0d333f8d-b877-4374-90a4-f78d1e97a74f/kube-rbac-proxy/0.log" Jul 16 11:13:40.536412 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:40.536384 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tds2p_dcc93a23-38c7-4c82-a82a-145238a435ed/dns-node-resolver/0.log" Jul 16 11:13:41.199575 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:41.199549 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qpkls_6bf38d4a-6103-4faa-be64-575b6953c111/node-ca/0.log" Jul 16 11:13:42.199927 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:42.199882 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mztsf_f3ff2214-1499-405a-af0d-f397817cc5d9/istio-proxy/0.log" Jul 16 11:13:42.307901 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:42.307863 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6nsj5_19b1da1f-e048-4802-8b67-7587c279e3ff/discovery/0.log" Jul 16 11:13:42.571700 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:42.571672 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f8555469b-j5x5d_186d512c-ec3f-4afd-b5fe-27037435710e/router/0.log" Jul 16 11:13:43.381872 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:43.381839 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kzxwn_a7efd23e-0880-4116-a8f7-883df2499fa8/serve-healthcheck-canary/0.log" Jul 16 11:13:43.917039 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:43.917011 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-vs4xw_707e6bfc-4cad-4b9e-9712-93f5e1970cf1/insights-operator/1.log" Jul 16 11:13:43.917205 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:43.917071 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-vs4xw_707e6bfc-4cad-4b9e-9712-93f5e1970cf1/insights-operator/0.log" Jul 16 11:13:44.018896 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:44.018871 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp9mw_6d768ed5-cd3f-4127-abdc-cc533901dfd4/kube-rbac-proxy/0.log" Jul 16 11:13:44.048637 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:44.048614 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp9mw_6d768ed5-cd3f-4127-abdc-cc533901dfd4/exporter/0.log" Jul 16 11:13:44.066810 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:44.066781 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-8cb67" Jul 16 11:13:44.073203 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:44.073185 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp9mw_6d768ed5-cd3f-4127-abdc-cc533901dfd4/extractor/0.log" Jul 16 11:13:47.386448 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:47.386426 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-cmdh2_574cbc13-8e5e-47d1-a0ab-c4ca563a5a95/manager/0.log" Jul 16 11:13:47.419839 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:47.419815 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-cmdh2_574cbc13-8e5e-47d1-a0ab-c4ca563a5a95/copy-manifests/0.log" Jul 16 11:13:47.552685 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:47.552660 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mztsf_f3ff2214-1499-405a-af0d-f397817cc5d9/istio-proxy/0.log" Jul 16 11:13:47.684538 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:47.684463 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6nsj5_19b1da1f-e048-4802-8b67-7587c279e3ff/discovery/0.log" Jul 16 11:13:47.916526 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:47.916500 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f8555469b-j5x5d_186d512c-ec3f-4afd-b5fe-27037435710e/router/0.log" Jul 16 11:13:49.169004 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:49.168923 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-85564789c6-7xlpz_ca6fbf6b-b6e2-4aab-9da1-1dd48485aa3c/manager/0.log" Jul 16 11:13:49.191704 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:49.191680 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mztsf_f3ff2214-1499-405a-af0d-f397817cc5d9/istio-proxy/0.log" Jul 16 11:13:49.294374 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:49.294336 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6nsj5_19b1da1f-e048-4802-8b67-7587c279e3ff/discovery/0.log" Jul 16 11:13:49.551080 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:49.550975 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f8555469b-j5x5d_186d512c-ec3f-4afd-b5fe-27037435710e/router/0.log" Jul 16 11:13:54.891645 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:54.891599 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-v8v66_e11cee50-17ab-4283-a72b-50a08cd6946b/kube-storage-version-migrator-operator/1.log" Jul 16 11:13:54.892674 ip-10-0-138-186 kubenswrapper[2568]: I0716 11:13:54.892651 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-v8v66_e11cee50-17ab-4283-a72b-50a08cd6946b/kube-storage-version-migrator-operator/0.log"