Jul 14 11:06:48.913397 ip-10-0-135-172 systemd[1]: Starting Kubernetes Kubelet... Jul 14 11:06:49.253248 ip-10-0-135-172 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:06:49.253248 ip-10-0-135-172 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 11:06:49.253248 ip-10-0-135-172 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:06:49.253248 ip-10-0-135-172 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 11:06:49.253248 ip-10-0-135-172 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:06:49.254094 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.254002 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 11:06:49.259358 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259336 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:06:49.259358 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259355 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:06:49.259358 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259359 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:06:49.259358 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259363 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:06:49.259358 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259367 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259371 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259376 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259381 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259384 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259387 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259390 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259393 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259396 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259399 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259401 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259404 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259407 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259410 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259412 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259417 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259421 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259425 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259428 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:06:49.259603 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259431 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259434 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259437 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259447 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259450 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259453 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259455 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259458 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259461 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259464 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259466 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259469 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259472 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259474 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259477 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259481 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259484 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259487 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259489 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:06:49.260081 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259492 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259495 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259498 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259500 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259503 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259506 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259508 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259511 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259513 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259516 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259519 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259522 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259525 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259527 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259530 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259533 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259535 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259538 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259541 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259543 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:06:49.260594 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259546 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259549 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259551 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259554 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259557 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259559 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259562 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259565 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259568 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259572 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259575 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259578 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259582 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259586 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259590 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259592 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259596 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259599 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259602 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:06:49.261110 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259605 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259608 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259612 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259615 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259617 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259621 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259624 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259627 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259630 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259632 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259635 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259638 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259641 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259643 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259646 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259649 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259651 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.259654 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260407 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260414 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:06:49.261613 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260417 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260422 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260427 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260431 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260435 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260438 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260441 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260444 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260446 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260451 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260455 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260458 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260461 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260464 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260467 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260471 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260474 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260477 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260480 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:06:49.262136 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260483 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260486 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260489 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260492 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260496 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260499 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260503 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260506 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260508 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260511 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260514 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260516 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260519 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260522 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260525 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260528 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260531 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260533 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260536 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260539 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:06:49.262675 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260541 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260544 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260547 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260550 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260552 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260555 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260557 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260560 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260563 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260566 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260568 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260571 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260574 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260576 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260579 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260582 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260584 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260587 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260590 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260592 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:06:49.263212 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260595 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260598 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260603 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260606 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260608 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260611 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260615 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260618 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260621 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260623 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260627 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260629 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260632 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260634 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260637 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260640 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260642 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260646 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260650 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260653 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:06:49.263741 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260656 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260659 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260663 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260666 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260669 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260671 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260674 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260677 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260680 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260682 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260685 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260688 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260691 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260693 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260696 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260699 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260702 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.260704 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261081 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261087 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:06:49.264288 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261090 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261093 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261096 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261099 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261102 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261105 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261107 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261113 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261116 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261119 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261122 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261125 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261128 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261131 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261134 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261139 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261143 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261146 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261150 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:06:49.264810 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261154 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261158 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261161 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261164 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261167 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261170 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261174 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261176 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261179 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261182 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261184 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261202 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261206 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261209 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261212 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261215 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261217 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261220 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261224 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261226 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261231 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:06:49.265306 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261233 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261236 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261239 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261242 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261245 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261248 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261250 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261254 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261257 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261259 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261262 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261265 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261267 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261270 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261273 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261275 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261278 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261281 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261284 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:06:49.265858 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261287 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261289 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261292 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261295 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261298 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261301 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261303 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261306 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261308 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261311 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261314 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261316 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261322 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261325 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261328 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261330 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261333 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261336 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261339 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:06:49.266371 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261342 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261345 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261348 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261351 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261353 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261356 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261359 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261361 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261364 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261367 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261369 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261372 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261378 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261381 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261383 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261386 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261389 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261391 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.261394 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261465 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 11:06:49.266866 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261472 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261479 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261483 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261488 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261492 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261497 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261502 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261505 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261508 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261512 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261516 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261519 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261522 2581 flags.go:64] FLAG: --cgroup-root="" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261525 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261528 2581 flags.go:64] FLAG: --client-ca-file="" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261532 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261536 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261540 2581 flags.go:64] FLAG: --cluster-domain="" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261543 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261546 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261549 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261553 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261556 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 11:06:49.267392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261559 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261564 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261567 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261571 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261574 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261577 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261580 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261583 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261590 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261594 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261597 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261600 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261603 2581 flags.go:64] FLAG: --enable-server="true" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261606 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261613 2581 flags.go:64] FLAG: --event-burst="100" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261616 2581 flags.go:64] FLAG: --event-qps="50" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261620 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261623 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261626 2581 flags.go:64] FLAG: --eviction-hard="" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261631 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261634 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261637 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261640 2581 flags.go:64] FLAG: --eviction-soft="" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261643 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261646 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 11:06:49.268023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261652 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261656 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261659 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261662 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261665 2581 flags.go:64] FLAG: --feature-gates="" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261669 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261672 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261675 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261679 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261682 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261685 2581 flags.go:64] FLAG: --help="false" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261688 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261692 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261695 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261698 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261701 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261705 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261708 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261711 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261714 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261717 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261721 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261724 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261728 2581 flags.go:64] FLAG: --kube-reserved="" Jul 14 11:06:49.268739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261731 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261734 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261737 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261740 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261743 2581 flags.go:64] FLAG: --lock-file="" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261746 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261749 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261752 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261757 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261761 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261764 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261767 2581 flags.go:64] FLAG: --logging-format="text" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261770 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261774 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261777 2581 flags.go:64] FLAG: --manifest-url="" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261780 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261784 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261787 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261791 2581 flags.go:64] FLAG: --max-pods="110" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261794 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261798 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261801 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261804 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261808 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261811 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 11:06:49.269375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261814 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261821 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261824 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261827 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261832 2581 flags.go:64] FLAG: --pod-cidr="" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261836 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261841 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261844 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261847 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261850 2581 flags.go:64] FLAG: --port="10250" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261854 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261857 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01a98d7f05026a741" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261860 2581 flags.go:64] FLAG: --qos-reserved="" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261864 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261867 2581 flags.go:64] FLAG: --register-node="true" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261870 2581 flags.go:64] FLAG: --register-with-taints="" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261874 2581 flags.go:64] FLAG: --registry-burst="10" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261877 2581 flags.go:64] FLAG: --registry-qps="5" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261880 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261883 2581 flags.go:64] FLAG: --reserved-memory="" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261887 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261890 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261893 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261896 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261899 2581 flags.go:64] FLAG: --runonce="false" Jul 14 11:06:49.270041 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261902 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261905 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261908 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261911 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261914 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261918 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261921 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261924 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261927 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261930 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261933 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261937 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261940 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261943 2581 flags.go:64] FLAG: --system-cgroups="" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261946 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261952 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261955 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261958 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261962 2581 flags.go:64] FLAG: --tls-min-version="" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261965 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261968 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261971 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261974 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261977 2581 flags.go:64] FLAG: --v="2" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261981 2581 flags.go:64] FLAG: --version="false" Jul 14 11:06:49.270708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261986 2581 flags.go:64] FLAG: --vmodule="" Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261989 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.261993 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262118 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262124 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262128 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262131 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262134 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262136 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262139 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262142 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262145 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262150 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262153 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262156 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262159 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262162 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262165 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262169 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262172 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:06:49.271372 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262175 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262177 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262180 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262183 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262185 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262203 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262209 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262214 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262218 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262221 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262224 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262227 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262230 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262234 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262237 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262240 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262243 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262247 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262250 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:06:49.271911 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262253 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262256 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262259 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262262 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262265 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262267 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262270 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262273 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262276 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262279 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262281 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262286 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262289 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262292 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262295 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262298 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262300 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262303 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262306 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262308 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:06:49.272423 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262311 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262314 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262316 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262319 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262322 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262325 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262327 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262332 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262335 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262337 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262342 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262345 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262348 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262350 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262353 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262356 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262359 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262362 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262364 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262367 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:06:49.273073 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262370 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262373 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262376 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262380 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262383 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262385 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262388 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262391 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262394 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262397 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262399 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262402 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262405 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262408 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262410 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262413 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262416 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262418 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262422 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262424 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:06:49.273615 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262428 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262431 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.262435 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.263048 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.269535 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.269549 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.269569 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.269575 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269631 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269636 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269640 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269644 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269648 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269651 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269654 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:06:49.274152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269657 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269660 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269663 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269665 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269668 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269671 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269674 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269677 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269679 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269682 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269685 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269688 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269691 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269693 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269696 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269699 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269701 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269704 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269708 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:06:49.274583 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269712 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269715 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269718 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269723 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269726 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269729 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269732 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269735 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269738 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269741 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269744 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269747 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269749 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269752 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269755 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269758 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269761 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269764 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269766 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:06:49.275086 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269769 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269771 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269775 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269777 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269780 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269783 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269785 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269788 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269791 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269793 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269796 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269799 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269801 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269804 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269807 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269810 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269814 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269819 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269822 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269825 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:06:49.275621 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269828 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269830 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269833 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269836 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269839 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269841 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269844 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269847 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269850 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269852 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269857 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269860 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269863 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269866 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269869 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269872 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269875 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269877 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269880 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269883 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:06:49.276152 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269886 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269888 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269891 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269893 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269896 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269899 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269902 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269905 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269908 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269910 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269913 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269916 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269919 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.269921 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:06:49.276836 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.269927 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270041 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270047 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270050 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270053 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270056 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270060 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270063 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270066 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270069 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270072 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270075 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270078 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270080 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270083 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270086 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270089 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270092 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270095 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270097 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:06:49.277479 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270100 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270103 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270106 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270108 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270111 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270114 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270117 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270120 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270122 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270125 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270128 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270131 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270133 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270136 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270139 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270141 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270144 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270147 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270149 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270152 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:06:49.278067 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270155 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270157 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270160 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270163 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270166 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270168 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270171 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270173 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270176 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270179 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270181 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270184 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270200 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270203 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270206 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270209 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270211 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270214 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270217 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270220 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:06:49.278656 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270223 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270227 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270232 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270235 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270238 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270241 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270244 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270246 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270249 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270252 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270254 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270257 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270259 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270262 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270265 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270267 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270270 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270273 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270275 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:06:49.279317 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270278 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270280 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270283 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270286 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270288 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270291 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270293 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270296 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270298 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270301 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270304 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270307 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270312 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270316 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270319 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270322 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270325 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270328 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:06:49.279816 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270331 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270335 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:49.270339 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.270344 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.270440 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.272567 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.273719 2581 server.go:1019] "Starting client certificate rotation" Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.273828 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.273894 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.275877 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 11:06:49.280308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.279216 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 11:06:49.292480 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.292455 2581 log.go:25] "Validated CRI v1 runtime API" Jul 14 11:06:49.294406 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.294391 2581 log.go:25] "Validated CRI v1 image API" Jul 14 11:06:49.295192 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.295167 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 11:06:49.297571 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.297551 2581 fs.go:135] Filesystem UUIDs: map[271724fe-85d6-41bd-b136-72a867950e99:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f15fa45c-9f19-4a73-bfc9-b93272920c84:/dev/nvme0n1p3] Jul 14 11:06:49.297639 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.297570 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 11:06:49.302803 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.302688 2581 manager.go:216] Machine: {Timestamp:2026-07-14 11:06:49.301641397 +0000 UTC m=+0.284612225 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099332 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25701ada368c4c0047d66f2a5dd3b1 SystemUUID:ec25701a-da36-8c4c-0047-d66f2a5dd3b1 BootID:ff50e649-0586-453d-9c22-dc3d3c89c6be Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:3b:67:d0:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:3b:67:d0:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:c0:6b:15:2c:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 11:06:49.302803 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.302795 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 11:06:49.302936 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.302871 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 11:06:49.303890 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.303867 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 11:06:49.304022 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.303893 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-172.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 11:06:49.304508 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.304493 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 11:06:49.304560 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.304552 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 11:06:49.304600 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.304563 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 11:06:49.304600 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.304572 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 11:06:49.305101 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.305089 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 11:06:49.306271 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.306260 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 11:06:49.309091 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309079 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 14 11:06:49.309238 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309228 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 11:06:49.309277 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309260 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 14 11:06:49.309306 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309285 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 11:06:49.309338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309308 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 11:06:49.309338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309322 2581 kubelet.go:398] "Adding apiserver pod source" Jul 14 11:06:49.309338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309336 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 11:06:49.309724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309712 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 11:06:49.309768 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.309744 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 11:06:49.311879 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.311851 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 11:06:49.313644 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313630 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 11:06:49.313694 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313662 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 11:06:49.313838 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313827 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 11:06:49.313882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313843 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 11:06:49.313882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313849 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 11:06:49.313882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313855 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 11:06:49.313882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313864 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 11:06:49.313882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.313869 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 11:06:49.314223 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314214 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 11:06:49.314223 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314223 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 11:06:49.314287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314231 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 11:06:49.314287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314237 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 11:06:49.314287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314246 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 11:06:49.314287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314254 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 11:06:49.314862 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314852 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 11:06:49.314862 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.314861 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 11:06:49.318136 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.318121 2581 server.go:1268] "Started kubelet" Jul 14 11:06:49.318365 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.318343 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 11:06:49.320770 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.318232 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 11:06:49.320865 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.320778 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 11:06:49.321451 ip-10-0-135-172 systemd[1]: Started Kubernetes Kubelet. Jul 14 11:06:49.323532 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.321932 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 11:06:49.323532 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.322104 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 11:06:49.323532 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.322357 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-172.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 11:06:49.323532 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.322901 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 11:06:49.324385 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.324349 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 11:06:49.324385 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.324370 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 14 11:06:49.327392 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.326559 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-172.ec2.internal.18c2234b71095741 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-172.ec2.internal,UID:ip-10-0-135-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-172.ec2.internal,},FirstTimestamp:2026-07-14 11:06:49.318094657 +0000 UTC m=+0.301065486,LastTimestamp:2026-07-14 11:06:49.318094657 +0000 UTC m=+0.301065486,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-172.ec2.internal,}" Jul 14 11:06:49.328901 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.328882 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 11:06:49.328988 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.328905 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 11:06:49.328988 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.328954 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.329586 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329566 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 11:06:49.329586 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329570 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 11:06:49.329730 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329596 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 11:06:49.329730 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329690 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 14 11:06:49.329730 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329701 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 14 11:06:49.329864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329730 2581 factory.go:55] Registering systemd factory Jul 14 11:06:49.329864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329746 2581 factory.go:223] Registration of the systemd container factory successfully Jul 14 11:06:49.329980 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329969 2581 factory.go:153] Registering CRI-O factory Jul 14 11:06:49.330014 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.329981 2581 factory.go:223] Registration of the crio container factory successfully Jul 14 11:06:49.330043 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.330024 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 11:06:49.330043 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.330043 2581 factory.go:103] Registering Raw factory Jul 14 11:06:49.330105 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.330053 2581 manager.go:1196] Started watching for new ooms in manager Jul 14 11:06:49.330504 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.330490 2581 manager.go:349] Starting recovery of all containers Jul 14 11:06:49.338107 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.337920 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-flk9x" Jul 14 11:06:49.341062 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.341034 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 11:06:49.341062 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.341059 2581 manager.go:354] Recovery completed Jul 14 11:06:49.341235 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.341209 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 11:06:49.342878 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.342855 2581 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 14 11:06:49.346561 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.346530 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.346636 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.346619 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-flk9x" Jul 14 11:06:49.348453 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.348436 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.348531 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.348468 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.348531 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.348485 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.349520 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.349502 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 11:06:49.349625 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.349614 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 11:06:49.349704 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.349696 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 14 11:06:49.349924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.349572 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.350804 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.350782 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.350899 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.350816 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.350899 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.350838 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.351519 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.351500 2581 policy_none.go:49] "None policy: Start" Jul 14 11:06:49.351595 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.351524 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 11:06:49.351595 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.351541 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 11:06:49.352635 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.352623 2581 policy_none.go:47] "Start" Jul 14 11:06:49.391764 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.391748 2581 manager.go:337] "Starting Device Plugin manager" Jul 14 11:06:49.391871 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.391784 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 11:06:49.391871 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.391797 2581 server.go:87] "Starting device plugin registration server" Jul 14 11:06:49.392044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.392031 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 11:06:49.392108 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.392048 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 11:06:49.392176 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.392157 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 11:06:49.392262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.392249 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 11:06:49.392262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.392261 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 11:06:49.392843 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.392818 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 11:06:49.392931 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.392875 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.484285 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.484254 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 11:06:49.485442 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.485424 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 11:06:49.485524 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.485461 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 11:06:49.485524 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.485484 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 11:06:49.485524 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.485516 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 11:06:49.488093 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.488069 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:06:49.492498 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.492483 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.493291 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.493274 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.493354 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.493310 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.493354 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.493327 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.493354 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.493355 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.503031 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.503017 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.503086 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.503037 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-172.ec2.internal\": node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.521102 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.521052 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.587430 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.587406 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"] Jul 14 11:06:49.587525 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.587481 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.588758 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.588744 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.588860 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.588775 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.588860 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.588790 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.590113 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590098 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.590228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590213 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.590277 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590242 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.590775 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590757 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.590850 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590759 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.590850 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590807 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.590850 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590818 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.590991 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590788 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.590991 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.590883 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.592528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.592147 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.592528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.592174 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:06:49.592952 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.592937 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:06:49.593005 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.592964 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:06:49.593005 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.592974 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:06:49.616842 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.616817 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-172.ec2.internal\" not found" node="ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.621348 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.621329 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.621451 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.621437 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-172.ec2.internal\" not found" node="ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.722429 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.722411 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.731408 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.731389 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9437156ea6019366c0c95106c22b0536-config\") pod \"kube-apiserver-proxy-ip-10-0-135-172.ec2.internal\" (UID: \"9437156ea6019366c0c95106c22b0536\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.731466 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.731416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f8244d1cc980c2b9527d65871244276-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"8f8244d1cc980c2b9527d65871244276\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.731466 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.731434 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8244d1cc980c2b9527d65871244276-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"8f8244d1cc980c2b9527d65871244276\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.823239 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.823180 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.832200 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.832180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f8244d1cc980c2b9527d65871244276-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"8f8244d1cc980c2b9527d65871244276\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.832266 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.832206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8244d1cc980c2b9527d65871244276-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"8f8244d1cc980c2b9527d65871244276\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.832266 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.832224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9437156ea6019366c0c95106c22b0536-config\") pod \"kube-apiserver-proxy-ip-10-0-135-172.ec2.internal\" (UID: \"9437156ea6019366c0c95106c22b0536\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.832266 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.832249 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9437156ea6019366c0c95106c22b0536-config\") pod \"kube-apiserver-proxy-ip-10-0-135-172.ec2.internal\" (UID: \"9437156ea6019366c0c95106c22b0536\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.832395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.832270 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f8244d1cc980c2b9527d65871244276-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"8f8244d1cc980c2b9527d65871244276\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.832395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.832295 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f8244d1cc980c2b9527d65871244276-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"8f8244d1cc980c2b9527d65871244276\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.920664 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.920631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:49.923322 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:49.923298 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:49.924733 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:49.924640 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" Jul 14 11:06:50.024128 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:50.024090 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:50.125071 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:50.124994 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:50.196366 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.196341 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:06:50.225161 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:50.225138 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:50.274687 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.274660 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 11:06:50.275217 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.274813 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 11:06:50.275217 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.274828 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 11:06:50.326296 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:50.326274 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found" Jul 14 11:06:50.350386 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.350352 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 11:01:49 +0000 UTC" deadline="2028-02-24 09:20:56.134595805 +0000 UTC" Jul 14 11:06:50.350386 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.350379 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14158h14m5.78421935s" Jul 14 11:06:50.394493 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.394441 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:06:50.429684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.429662 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" Jul 14 11:06:50.440787 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.440768 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 11:06:50.442518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.442503 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" Jul 14 11:06:50.450684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.450666 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 11:06:50.526842 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:50.526810 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8244d1cc980c2b9527d65871244276.slice/crio-37575ede7d3952b805413f5c87cb9fe4456eb7641c3cf84700298b3e059e38cd WatchSource:0}: Error finding container 37575ede7d3952b805413f5c87cb9fe4456eb7641c3cf84700298b3e059e38cd: Status 404 returned error can't find the container with id 37575ede7d3952b805413f5c87cb9fe4456eb7641c3cf84700298b3e059e38cd Jul 14 11:06:50.527284 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:50.527265 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9437156ea6019366c0c95106c22b0536.slice/crio-cb8a3ca5f7a24d029eee75bf190d679d4676f83ceb2d474472d6b2cecb353515 WatchSource:0}: Error finding container cb8a3ca5f7a24d029eee75bf190d679d4676f83ceb2d474472d6b2cecb353515: Status 404 returned error can't find the container with id cb8a3ca5f7a24d029eee75bf190d679d4676f83ceb2d474472d6b2cecb353515 Jul 14 11:06:50.534144 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.534126 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:06:50.709470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:50.709066 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:06:51.310422 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.310373 2581 apiserver.go:52] "Watching apiserver" Jul 14 11:06:51.315835 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.315794 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 11:06:51.316881 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.316853 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv","openshift-dns/node-resolver-9rqsm","openshift-multus/multus-additional-cni-plugins-j2466","openshift-network-operator/iptables-alerter-vngvw","openshift-ovn-kubernetes/ovnkube-node-b2w2v","openshift-cluster-node-tuning-operator/tuned-rd8k8","openshift-image-registry/node-ca-9n7wv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal","openshift-multus/multus-69sr4","openshift-multus/network-metrics-daemon-tp4pg","openshift-network-diagnostics/network-check-target-zc6px","kube-system/global-pull-secret-syncer-mwx66","kube-system/konnectivity-agent-wlfnm"] Jul 14 11:06:51.318565 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.318541 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.319925 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.319724 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.319925 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.319811 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:06:51.320906 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.320759 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.321277 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.321207 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.321411 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.321351 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pxxgl\"" Jul 14 11:06:51.321619 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.321606 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.323405 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.323383 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-46zxz\"" Jul 14 11:06:51.323506 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.323410 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.323654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.323637 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.325118 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.324835 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.326624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.326202 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.326624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.326347 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.327089 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.327153 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.327315 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.327529 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.327702 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-snbgk\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.328020 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 11:06:51.328155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.328113 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.328777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.328233 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.328777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.328338 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.328777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.328741 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rbrh7\"" Jul 14 11:06:51.329373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.329252 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.330705 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.330417 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 11:06:51.332768 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.332654 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 11:06:51.332992 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.332969 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 11:06:51.333853 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.333832 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.333948 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.333907 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 11:06:51.334036 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.333837 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.334230 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.334187 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 11:06:51.334297 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.334282 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 11:06:51.334648 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.334624 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 11:06:51.334728 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.334692 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x24zx\"" Jul 14 11:06:51.334887 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.334868 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.335069 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.335050 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.336622 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.336600 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p74nz\"" Jul 14 11:06:51.336744 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.336724 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.336860 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.336841 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.337754 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.337721 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2cmcm\"" Jul 14 11:06:51.338052 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.338022 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 11:06:51.338174 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.338140 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gqfpc\"" Jul 14 11:06:51.339210 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.338786 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.339452 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.338786 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:51.339565 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.339533 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:06:51.339565 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.338969 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 11:06:51.339681 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339659 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 11:06:51.339681 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.339789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339701 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f196ae66-b86f-455a-b491-e7998e88bbea-tmp\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.339789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm2sx\" (UniqueName: \"kubernetes.io/projected/f196ae66-b86f-455a-b491-e7998e88bbea-kube-api-access-pm2sx\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.339789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339742 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 11:06:51.339789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339768 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-log-socket\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.339984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339795 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xqss4\"" Jul 14 11:06:51.339984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339850 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-env-overrides\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.339984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339882 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-var-lib-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.339984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.339933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.340204 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340015 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovn-node-metrics-cert\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.340749 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340634 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:51.340749 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.340709 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:06:51.340749 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovnkube-script-lib\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.340924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340778 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-serviceca\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.340924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-kubernetes\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.340924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-systemd\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.340924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-host\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.340924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gxn6\" (UniqueName: \"kubernetes.io/projected/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-kube-api-access-9gxn6\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.340924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340904 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysctl-conf\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340928 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9270024d-9198-42d3-b50f-a38e6d29db56-iptables-alerter-script\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.340982 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-kubelet\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-cni-netd\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341030 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-lib-modules\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f196ae66-b86f-455a-b491-e7998e88bbea-etc-tuned\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341123 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-systemd-units\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341162 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341188 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-ovn\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6xw8\" (UniqueName: \"kubernetes.io/projected/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-kube-api-access-t6xw8\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341273 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cni-binary-copy\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341300 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9270024d-9198-42d3-b50f-a38e6d29db56-host-slash\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341324 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-etc-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341349 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-node-log\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysconfig\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-systemd\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-host\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60660683-0b39-4db6-a3ad-0261e7a7a741-kubelet-config\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-hosts-file\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341644 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmq2\" (UniqueName: \"kubernetes.io/projected/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-kube-api-access-4pmq2\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341669 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-var-lib-kubelet\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341810 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60660683-0b39-4db6-a3ad-0261e7a7a741-dbus\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.341844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-tmp-dir\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovnkube-config\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-modprobe-d\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.341998 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1b7bed04-1028-4f4d-8a65-03347daeef1a-agent-certs\") pod \"konnectivity-agent-wlfnm\" (UID: \"1b7bed04-1028-4f4d-8a65-03347daeef1a\") " pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342037 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1b7bed04-1028-4f4d-8a65-03347daeef1a-konnectivity-ca\") pod \"konnectivity-agent-wlfnm\" (UID: \"1b7bed04-1028-4f4d-8a65-03347daeef1a\") " pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342084 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-os-release\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-run-netns\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342157 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-cni-bin\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cnibin\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm5sj\" (UniqueName: \"kubernetes.io/projected/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-kube-api-access-pm5sj\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342264 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-slash\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4ftg\" (UniqueName: \"kubernetes.io/projected/9270024d-9198-42d3-b50f-a38e6d29db56-kube-api-access-p4ftg\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysctl-d\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342427 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-run\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.342492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-sys\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.343279 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.342485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-system-cni-dir\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.344343 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.344304 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 11:06:51.375813 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.375789 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4h4kc" Jul 14 11:06:51.381463 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.381031 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4h4kc" Jul 14 11:06:51.430722 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.430702 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 11:06:51.442728 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm2sx\" (UniqueName: \"kubernetes.io/projected/f196ae66-b86f-455a-b491-e7998e88bbea-kube-api-access-pm2sx\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.442856 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442747 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-log-socket\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.442856 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442776 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-env-overrides\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.442856 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442809 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-os-release\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.442856 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-var-lib-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442861 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbb2c\" (UniqueName: \"kubernetes.io/projected/e27439e8-3bca-445a-86ed-d880b7684c0b-kube-api-access-dbb2c\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442886 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-log-socket\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442937 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovn-node-metrics-cert\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.442978 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-var-lib-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovnkube-script-lib\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443068 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-serviceca\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-kubernetes\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-conf-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.443161 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443149 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-systemd\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443216 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-host\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gxn6\" (UniqueName: \"kubernetes.io/projected/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-kube-api-access-9gxn6\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443266 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysctl-conf\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443292 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9270024d-9198-42d3-b50f-a38e6d29db56-iptables-alerter-script\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-socket-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443395 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-kubelet\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-etc-selinux\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443434 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-etc-kubernetes\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.443696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443547 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-env-overrides\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443722 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysctl-conf\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443810 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-kubelet\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443877 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-systemd\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443930 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-kubernetes\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.443972 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-cni-netd\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444022 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-host\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444037 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-lib-modules\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f196ae66-b86f-455a-b491-e7998e88bbea-etc-tuned\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-registration-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444157 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444219 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9270024d-9198-42d3-b50f-a38e6d29db56-iptables-alerter-script\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444236 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-cni-netd\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-k8s-cni-cncf-io\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444223 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-serviceca\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444273 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.444352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-cni-bin\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444352 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-lib-modules\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444451 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-multus-certs\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-systemd-units\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovnkube-script-lib\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-ovn\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444648 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6xw8\" (UniqueName: \"kubernetes.io/projected/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-kube-api-access-t6xw8\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444662 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-systemd-units\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-cnibin\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444707 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-ovn\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-cni-multus\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cni-binary-copy\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444831 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9270024d-9198-42d3-b50f-a38e6d29db56-host-slash\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444855 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-etc-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444877 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-node-log\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.445287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysconfig\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-systemd\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-host\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.444988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e27439e8-3bca-445a-86ed-d880b7684c0b-cni-binary-copy\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445032 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60660683-0b39-4db6-a3ad-0261e7a7a741-kubelet-config\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-hosts-file\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445088 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmq2\" (UniqueName: \"kubernetes.io/projected/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-kube-api-access-4pmq2\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445126 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-var-lib-kubelet\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445156 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445152 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6z4m\" (UniqueName: \"kubernetes.io/projected/4e1b73a1-619f-41c5-8a62-4380a23919cf-kube-api-access-m6z4m\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445356 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-hosts-file\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445384 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9270024d-9198-42d3-b50f-a38e6d29db56-host-slash\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445402 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60660683-0b39-4db6-a3ad-0261e7a7a741-kubelet-config\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60660683-0b39-4db6-a3ad-0261e7a7a741-dbus\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-node-log\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.446307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-tmp-dir\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445502 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysconfig\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445517 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovnkube-config\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-modprobe-d\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445566 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1b7bed04-1028-4f4d-8a65-03347daeef1a-agent-certs\") pod \"konnectivity-agent-wlfnm\" (UID: \"1b7bed04-1028-4f4d-8a65-03347daeef1a\") " pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445572 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-var-lib-kubelet\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1b7bed04-1028-4f4d-8a65-03347daeef1a-konnectivity-ca\") pod \"konnectivity-agent-wlfnm\" (UID: \"1b7bed04-1028-4f4d-8a65-03347daeef1a\") " pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445679 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-systemd\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-device-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445770 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60660683-0b39-4db6-a3ad-0261e7a7a741-dbus\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-os-release\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-sys-fs\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445916 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-etc-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.445966 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-tmp-dir\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446248 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-os-release\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446290 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovnkube-config\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446349 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-modprobe-d\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.447158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446376 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-host\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-cni-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446418 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-netns\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446456 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-kubelet\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-hostroot\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446552 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-run-netns\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-cni-bin\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446613 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cnibin\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm5sj\" (UniqueName: \"kubernetes.io/projected/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-kube-api-access-pm5sj\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-system-cni-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-socket-dir-parent\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446748 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cnibin\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446785 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-run-netns\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446828 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-cni-bin\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.446877 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-cni-binary-copy\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447022 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-slash\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447072 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-host-slash\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448011 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.447181 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.447245 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:51.94722399 +0000 UTC m=+2.930194810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447339 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1b7bed04-1028-4f4d-8a65-03347daeef1a-konnectivity-ca\") pod \"konnectivity-agent-wlfnm\" (UID: \"1b7bed04-1028-4f4d-8a65-03347daeef1a\") " pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-run-openvswitch\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447490 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4ftg\" (UniqueName: \"kubernetes.io/projected/9270024d-9198-42d3-b50f-a38e6d29db56-kube-api-access-p4ftg\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447527 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-daemon-config\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447603 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-ovn-node-metrics-cert\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysctl-d\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-run\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447727 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-run\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-sys\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447773 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-system-cni-dir\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447786 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-sys\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447814 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f196ae66-b86f-455a-b491-e7998e88bbea-etc-sysctl-d\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.448864 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447827 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz79g\" (UniqueName: \"kubernetes.io/projected/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-kube-api-access-dz79g\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.449735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447857 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-system-cni-dir\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.449735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447907 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f196ae66-b86f-455a-b491-e7998e88bbea-tmp\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.449735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.447996 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.449735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.448032 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f196ae66-b86f-455a-b491-e7998e88bbea-etc-tuned\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.450467 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.450441 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1b7bed04-1028-4f4d-8a65-03347daeef1a-agent-certs\") pod \"konnectivity-agent-wlfnm\" (UID: \"1b7bed04-1028-4f4d-8a65-03347daeef1a\") " pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.451904 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.451823 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gxn6\" (UniqueName: \"kubernetes.io/projected/8eb6e134-f251-4b16-8c61-9a8e76e30d4d-kube-api-access-9gxn6\") pod \"node-ca-9n7wv\" (UID: \"8eb6e134-f251-4b16-8c61-9a8e76e30d4d\") " pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.456331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.452542 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm2sx\" (UniqueName: \"kubernetes.io/projected/f196ae66-b86f-455a-b491-e7998e88bbea-kube-api-access-pm2sx\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.456331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.453092 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f196ae66-b86f-455a-b491-e7998e88bbea-tmp\") pod \"tuned-rd8k8\" (UID: \"f196ae66-b86f-455a-b491-e7998e88bbea\") " pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.456331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.453615 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmq2\" (UniqueName: \"kubernetes.io/projected/8e708d4a-7e79-4aae-b247-fbdea80ca0f5-kube-api-access-4pmq2\") pod \"node-resolver-9rqsm\" (UID: \"8e708d4a-7e79-4aae-b247-fbdea80ca0f5\") " pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.456331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.453652 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6xw8\" (UniqueName: \"kubernetes.io/projected/a0fa7ed7-bff7-4678-821d-60bd5ea115cd-kube-api-access-t6xw8\") pod \"ovnkube-node-b2w2v\" (UID: \"a0fa7ed7-bff7-4678-821d-60bd5ea115cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.456331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.455862 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4ftg\" (UniqueName: \"kubernetes.io/projected/9270024d-9198-42d3-b50f-a38e6d29db56-kube-api-access-p4ftg\") pod \"iptables-alerter-vngvw\" (UID: \"9270024d-9198-42d3-b50f-a38e6d29db56\") " pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.456331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.455960 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:06:51.457330 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.456970 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm5sj\" (UniqueName: \"kubernetes.io/projected/f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c-kube-api-access-pm5sj\") pod \"multus-additional-cni-plugins-j2466\" (UID: \"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c\") " pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.491318 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.491262 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" event={"ID":"9437156ea6019366c0c95106c22b0536","Type":"ContainerStarted","Data":"cb8a3ca5f7a24d029eee75bf190d679d4676f83ceb2d474472d6b2cecb353515"} Jul 14 11:06:51.492647 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.492622 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" event={"ID":"8f8244d1cc980c2b9527d65871244276","Type":"ContainerStarted","Data":"37575ede7d3952b805413f5c87cb9fe4456eb7641c3cf84700298b3e059e38cd"} Jul 14 11:06:51.548350 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548315 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz79g\" (UniqueName: \"kubernetes.io/projected/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-kube-api-access-dz79g\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548357 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-os-release\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548389 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbb2c\" (UniqueName: \"kubernetes.io/projected/e27439e8-3bca-445a-86ed-d880b7684c0b-kube-api-access-dbb2c\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-conf-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-socket-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548474 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-os-release\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-etc-selinux\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-etc-kubernetes\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-registration-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548598 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-etc-kubernetes\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548617 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-conf-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-k8s-cni-cncf-io\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-cni-bin\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548714 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-socket-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548725 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-k8s-cni-cncf-io\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548736 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-multus-certs\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548687 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-registration-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548714 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-etc-selinux\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-cnibin\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548774 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-cni-bin\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548801 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-cni-multus\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548819 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-cnibin\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548793 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-multus-certs\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.548877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548843 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-cni-multus\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548870 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e27439e8-3bca-445a-86ed-d880b7684c0b-cni-binary-copy\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6z4m\" (UniqueName: \"kubernetes.io/projected/4e1b73a1-619f-41c5-8a62-4380a23919cf-kube-api-access-m6z4m\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-device-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.548994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-sys-fs\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.549011 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-cni-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-netns\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.549067 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:06:52.049048111 +0000 UTC m=+3.032018929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549098 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-run-netns\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-kubelet\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549175 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-host-var-lib-kubelet\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549241 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-sys-fs\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549291 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-device-dir\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549311 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-cni-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.549833 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-hostroot\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549354 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-hostroot\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-system-cni-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-socket-dir-parent\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-daemon-config\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549424 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-system-cni-dir\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549490 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-socket-dir-parent\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e27439e8-3bca-445a-86ed-d880b7684c0b-cni-binary-copy\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.550724 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.549913 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e27439e8-3bca-445a-86ed-d880b7684c0b-multus-daemon-config\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.554673 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.554654 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:06:51.554789 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.554677 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:06:51.554789 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.554691 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:51.554789 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.554756 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:52.054742444 +0000 UTC m=+3.037713259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:51.557817 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.557788 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbb2c\" (UniqueName: \"kubernetes.io/projected/e27439e8-3bca-445a-86ed-d880b7684c0b-kube-api-access-dbb2c\") pod \"multus-69sr4\" (UID: \"e27439e8-3bca-445a-86ed-d880b7684c0b\") " pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.558731 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.558702 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz79g\" (UniqueName: \"kubernetes.io/projected/14a5cd3f-eae1-4b79-ba37-4ffbf5231281-kube-api-access-dz79g\") pod \"aws-ebs-csi-driver-node-xg5gv\" (UID: \"14a5cd3f-eae1-4b79-ba37-4ffbf5231281\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.558879 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.558857 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6z4m\" (UniqueName: \"kubernetes.io/projected/4e1b73a1-619f-41c5-8a62-4380a23919cf-kube-api-access-m6z4m\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:51.634725 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.634639 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" Jul 14 11:06:51.643791 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.643770 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9rqsm" Jul 14 11:06:51.646710 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:51.646680 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf196ae66_b86f_455a_b491_e7998e88bbea.slice/crio-bed26453f34dbaded2b27035baca6a2be65d4751add9102bce73b4014a3de95b WatchSource:0}: Error finding container bed26453f34dbaded2b27035baca6a2be65d4751add9102bce73b4014a3de95b: Status 404 returned error can't find the container with id bed26453f34dbaded2b27035baca6a2be65d4751add9102bce73b4014a3de95b Jul 14 11:06:51.653205 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:51.653131 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e708d4a_7e79_4aae_b247_fbdea80ca0f5.slice/crio-6e63e1946ecc772c4a7af3eeeb827d2cd29fc96ee6aaeb9490d8de5c380b28b3 WatchSource:0}: Error finding container 6e63e1946ecc772c4a7af3eeeb827d2cd29fc96ee6aaeb9490d8de5c380b28b3: Status 404 returned error can't find the container with id 6e63e1946ecc772c4a7af3eeeb827d2cd29fc96ee6aaeb9490d8de5c380b28b3 Jul 14 11:06:51.654277 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.654061 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" Jul 14 11:06:51.660338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.660320 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j2466" Jul 14 11:06:51.669216 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.669197 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vngvw" Jul 14 11:06:51.677055 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.677015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:06:51.688234 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.688203 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:06:51.695921 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.695901 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9n7wv" Jul 14 11:06:51.700992 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.700974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-69sr4" Jul 14 11:06:51.951544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:51.951461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:51.951674 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.951589 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:51.951674 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:51.951644 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:52.951629306 +0000 UTC m=+3.934600120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:52.011188 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:52.011146 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a5cd3f_eae1_4b79_ba37_4ffbf5231281.slice/crio-c2f9673644bcb3202927f661e124263127f6c3b2fa818109c79c98ef17adcf3c WatchSource:0}: Error finding container c2f9673644bcb3202927f661e124263127f6c3b2fa818109c79c98ef17adcf3c: Status 404 returned error can't find the container with id c2f9673644bcb3202927f661e124263127f6c3b2fa818109c79c98ef17adcf3c Jul 14 11:06:52.052345 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.052316 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:52.052484 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.052460 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:52.052533 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.052526 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:06:53.05250682 +0000 UTC m=+4.035477637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:52.153126 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.153077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:52.153306 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.153254 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:06:52.153306 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.153281 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:06:52.153306 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.153294 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:52.153463 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.153352 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:53.153331485 +0000 UTC m=+4.136302312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:52.384257 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.384112 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:01:51 +0000 UTC" deadline="2028-02-07 00:25:27.934171975 +0000 UTC" Jul 14 11:06:52.384257 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.384179 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13741h18m35.549997775s" Jul 14 11:06:52.486194 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.486169 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:52.486344 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.486304 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:06:52.496153 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.496103 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" event={"ID":"f196ae66-b86f-455a-b491-e7998e88bbea","Type":"ContainerStarted","Data":"bed26453f34dbaded2b27035baca6a2be65d4751add9102bce73b4014a3de95b"} Jul 14 11:06:52.497451 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.497416 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" event={"ID":"14a5cd3f-eae1-4b79-ba37-4ffbf5231281","Type":"ContainerStarted","Data":"c2f9673644bcb3202927f661e124263127f6c3b2fa818109c79c98ef17adcf3c"} Jul 14 11:06:52.499635 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.499601 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9rqsm" event={"ID":"8e708d4a-7e79-4aae-b247-fbdea80ca0f5","Type":"ContainerStarted","Data":"6e63e1946ecc772c4a7af3eeeb827d2cd29fc96ee6aaeb9490d8de5c380b28b3"} Jul 14 11:06:52.594525 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.594490 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:06:52.838352 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:52.838321 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27439e8_3bca_445a_86ed_d880b7684c0b.slice/crio-255c56fc8a49835b4f10adc1d0f3b6d9212d23e1a0b490dabb112323878cc508 WatchSource:0}: Error finding container 255c56fc8a49835b4f10adc1d0f3b6d9212d23e1a0b490dabb112323878cc508: Status 404 returned error can't find the container with id 255c56fc8a49835b4f10adc1d0f3b6d9212d23e1a0b490dabb112323878cc508 Jul 14 11:06:52.843025 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:52.842976 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fa7ed7_bff7_4678_821d_60bd5ea115cd.slice/crio-e93fe84ac1a76e139bb8af12b1f08b855bd1cb0e60de0c6fa9495185f5d714fb WatchSource:0}: Error finding container e93fe84ac1a76e139bb8af12b1f08b855bd1cb0e60de0c6fa9495185f5d714fb: Status 404 returned error can't find the container with id e93fe84ac1a76e139bb8af12b1f08b855bd1cb0e60de0c6fa9495185f5d714fb Jul 14 11:06:52.846732 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:52.846708 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9270024d_9198_42d3_b50f_a38e6d29db56.slice/crio-37eefa6c597bcb665c21872bfc97e98745155cddcf920357ca8bb86fec12e80a WatchSource:0}: Error finding container 37eefa6c597bcb665c21872bfc97e98745155cddcf920357ca8bb86fec12e80a: Status 404 returned error can't find the container with id 37eefa6c597bcb665c21872bfc97e98745155cddcf920357ca8bb86fec12e80a Jul 14 11:06:52.850209 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:06:52.850105 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7bed04_1028_4f4d_8a65_03347daeef1a.slice/crio-bbd4e163188674f987b76a376211230bb0108159a912979020309b7ab818bcec WatchSource:0}: Error finding container bbd4e163188674f987b76a376211230bb0108159a912979020309b7ab818bcec: Status 404 returned error can't find the container with id bbd4e163188674f987b76a376211230bb0108159a912979020309b7ab818bcec Jul 14 11:06:52.960934 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:52.960748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:52.961543 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.961103 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:52.961543 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:52.961189 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:54.961168899 +0000 UTC m=+5.944139723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:53.062404 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.061843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:53.062404 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.062009 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:53.062404 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.062069 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:06:55.062050426 +0000 UTC m=+6.045021242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:53.163005 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.162979 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:53.163168 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.163149 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:06:53.163235 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.163176 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:06:53.163235 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.163189 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:53.163338 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.163245 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:55.163225811 +0000 UTC m=+6.146196624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:53.385355 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.385294 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:01:51 +0000 UTC" deadline="2027-12-28 21:47:12.734730688 +0000 UTC" Jul 14 11:06:53.385355 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.385354 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12778h40m19.349379765s" Jul 14 11:06:53.490174 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.487045 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:53.490174 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.487188 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:06:53.490174 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.487571 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:53.490174 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:53.487669 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:06:53.505847 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.505358 2581 generic.go:356] "Generic (PLEG): container finished" podID="8f8244d1cc980c2b9527d65871244276" containerID="5353d7adc2e5c438e1cec780c07e910f832639e33f02f7f8aa692d59d70320ec" exitCode=0 Jul 14 11:06:53.505847 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.505444 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" event={"ID":"8f8244d1cc980c2b9527d65871244276","Type":"ContainerDied","Data":"5353d7adc2e5c438e1cec780c07e910f832639e33f02f7f8aa692d59d70320ec"} Jul 14 11:06:53.512104 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.512035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wlfnm" event={"ID":"1b7bed04-1028-4f4d-8a65-03347daeef1a","Type":"ContainerStarted","Data":"bbd4e163188674f987b76a376211230bb0108159a912979020309b7ab818bcec"} Jul 14 11:06:53.515376 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.515351 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vngvw" event={"ID":"9270024d-9198-42d3-b50f-a38e6d29db56","Type":"ContainerStarted","Data":"37eefa6c597bcb665c21872bfc97e98745155cddcf920357ca8bb86fec12e80a"} Jul 14 11:06:53.517798 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.517774 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerStarted","Data":"8d770b02afa455af3813326a1896ce52cfa3ac4db03448ddbe78f409f4297ee3"} Jul 14 11:06:53.525676 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.525594 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" event={"ID":"9437156ea6019366c0c95106c22b0536","Type":"ContainerStarted","Data":"6d0e6a2b3ca5716c0800c0fd15e3a7ec029b0d810052f71c86e27b3a0fb19e82"} Jul 14 11:06:53.534044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.533993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9n7wv" event={"ID":"8eb6e134-f251-4b16-8c61-9a8e76e30d4d","Type":"ContainerStarted","Data":"c1ca59ad1f846bb689ee79b46f123bb4e1ba5558c57929710c02ddf6af9ea4fc"} Jul 14 11:06:53.544317 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.544217 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"e93fe84ac1a76e139bb8af12b1f08b855bd1cb0e60de0c6fa9495185f5d714fb"} Jul 14 11:06:53.552925 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:53.552902 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-69sr4" event={"ID":"e27439e8-3bca-445a-86ed-d880b7684c0b","Type":"ContainerStarted","Data":"255c56fc8a49835b4f10adc1d0f3b6d9212d23e1a0b490dabb112323878cc508"} Jul 14 11:06:54.486066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:54.486018 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:54.486637 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:54.486175 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:06:54.580296 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:54.580264 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" event={"ID":"8f8244d1cc980c2b9527d65871244276","Type":"ContainerStarted","Data":"8578b7bc117da60815c30098497cf4e0e51c50f3fdaf2bd71b86451727563f19"} Jul 14 11:06:54.594771 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:54.594690 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" podStartSLOduration=4.594672194 podStartE2EDuration="4.594672194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:53.53927848 +0000 UTC m=+4.522249316" watchObservedRunningTime="2026-07-14 11:06:54.594672194 +0000 UTC m=+5.577643031" Jul 14 11:06:54.977813 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:54.977171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:54.977813 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:54.977363 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:54.977813 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:54.977424 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:58.977405354 +0000 UTC m=+9.960376170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:55.078509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:55.078162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:55.078509 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.078305 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:55.078509 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.078362 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:06:59.078343858 +0000 UTC m=+10.061314674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:55.178978 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:55.178942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:55.179161 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.179139 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:06:55.179243 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.179171 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:06:55.179243 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.179184 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:55.179367 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.179247 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:06:59.179226984 +0000 UTC m=+10.162197799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:55.491978 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:55.491362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:55.491978 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.491489 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:06:55.491978 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:55.491568 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:55.491978 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:55.491635 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:06:56.486150 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:56.486104 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:56.486327 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:56.486269 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:06:57.487127 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:57.486656 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:57.487127 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:57.486800 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:06:57.487644 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:57.487159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:57.487644 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:57.487264 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:06:58.485880 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:58.485844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:58.486067 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:58.485994 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:06:59.009799 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:59.009663 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:59.009799 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.009814 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:59.010345 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.009880 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:07.009860857 +0000 UTC m=+17.992831674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:06:59.111102 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:59.111058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:06:59.111329 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.111290 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:59.111421 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.111371 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:07:07.111349281 +0000 UTC m=+18.094320114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:06:59.211560 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:59.211526 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:59.211736 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.211717 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:06:59.211817 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.211744 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:06:59.211817 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.211758 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:59.211817 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.211817 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:07.211798957 +0000 UTC m=+18.194769772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:06:59.487201 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:59.486740 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:06:59.487201 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.486852 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:06:59.487415 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:06:59.487251 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:06:59.487415 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:06:59.487335 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:00.486000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:00.485671 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:00.486000 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:00.485821 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:01.485814 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:01.485778 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:01.486000 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:01.485924 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:01.486000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:01.485977 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:01.486381 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:01.486091 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:02.485876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:02.485839 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:02.486047 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:02.485974 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:03.486764 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:03.486689 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:03.486764 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:03.486723 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:03.487173 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:03.486792 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:03.487173 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:03.486905 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:04.485771 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:04.485742 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:04.485945 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:04.485848 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:05.489586 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:05.489555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:05.490015 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:05.489669 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:05.490015 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:05.489726 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:05.490015 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:05.489816 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:06.485654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:06.485625 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:06.485890 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:06.485737 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:07.072770 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:07.072734 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:07.073143 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.072842 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:07:07.073143 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.072901 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:23.072886583 +0000 UTC m=+34.055857396 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:07:07.173827 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:07.173794 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:07.173981 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.173933 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:07:07.174022 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.173994 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:07:23.173978762 +0000 UTC m=+34.156949574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:07:07.274435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:07.274345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:07.274611 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.274512 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:07:07.274611 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.274541 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:07:07.274611 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.274556 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:07:07.274739 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.274620 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:23.274601993 +0000 UTC m=+34.257572823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:07:07.486054 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:07.486013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:07.486242 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.486154 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:07.486308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:07.486238 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:07.486362 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:07.486339 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:08.485853 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:08.485814 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:08.486360 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:08.485967 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:09.486646 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:09.486592 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:09.487132 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:09.486699 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:09.487132 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:09.486753 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:09.487132 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:09.486833 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:10.486329 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:10.486296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:10.486511 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:10.486441 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:11.488866 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.487901 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:11.488866 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.488183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:11.488866 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:11.488307 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:11.488866 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:11.488359 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:11.612170 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.612137 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" event={"ID":"14a5cd3f-eae1-4b79-ba37-4ffbf5231281","Type":"ContainerStarted","Data":"e4699b7dba2f165b83a47095f19eec8c0b48e3ce0a7ffd27a47b7e6c88ee7337"} Jul 14 11:07:11.613260 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.613215 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" event={"ID":"f196ae66-b86f-455a-b491-e7998e88bbea","Type":"ContainerStarted","Data":"c38d56ef89a82b7db96e35d332b1ada3e8fc2bc50d0f3b122112dcbf601a81bd"} Jul 14 11:07:11.614698 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.614672 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9n7wv" event={"ID":"8eb6e134-f251-4b16-8c61-9a8e76e30d4d","Type":"ContainerStarted","Data":"ba51411621d84c709da542bc60a961bcda7a727657d2884d43eb4f2a0e5f16fe"} Jul 14 11:07:11.615898 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.615880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-69sr4" event={"ID":"e27439e8-3bca-445a-86ed-d880b7684c0b","Type":"ContainerStarted","Data":"96b9489aae3ea5e14bcbe2e601fd3c37e23016303eea71910aa4fa56489e131d"} Jul 14 11:07:11.617037 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.617019 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9rqsm" event={"ID":"8e708d4a-7e79-4aae-b247-fbdea80ca0f5","Type":"ContainerStarted","Data":"494819ec5f1c83256d69d328791a599ed7238de2b261372e9f71fbd3a747b3c5"} Jul 14 11:07:11.618246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.618223 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wlfnm" event={"ID":"1b7bed04-1028-4f4d-8a65-03347daeef1a","Type":"ContainerStarted","Data":"20b7c17923f18e843c0aa7bc3ded3b02d67582c641efcb57edcc76f631a59fe2"} Jul 14 11:07:11.619556 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.619535 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerStarted","Data":"3a53ca6ff2d8b89a7ef5ab48b85e6720d46c67bb980caf2f003cbc99a463ee57"} Jul 14 11:07:11.631572 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.631530 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rd8k8" podStartSLOduration=2.936881735 podStartE2EDuration="22.631516029s" totalImagesPullingTime="19.694634294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:11.631481685 +0000 UTC m=+22.614452521" watchObservedRunningTime="2026-07-14 11:07:11.631516029 +0000 UTC m=+22.614486865" Jul 14 11:07:11.631686 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.631660 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" podStartSLOduration=21.631656114 podStartE2EDuration="21.631656114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:06:54.595813179 +0000 UTC m=+5.578784016" watchObservedRunningTime="2026-07-14 11:07:11.631656114 +0000 UTC m=+22.614626949" Jul 14 11:07:11.646164 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.646090 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wlfnm" podStartSLOduration=4.164022532 podStartE2EDuration="22.646069562s" totalImagesPullingTime="18.48204703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:11.64573051 +0000 UTC m=+22.628701344" watchObservedRunningTime="2026-07-14 11:07:11.646069562 +0000 UTC m=+22.629040401" Jul 14 11:07:11.664434 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:11.664381 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-69sr4" podStartSLOduration=4.102113073 podStartE2EDuration="22.664367274s" totalImagesPullingTime="18.562254201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:11.66384191 +0000 UTC m=+22.646812758" watchObservedRunningTime="2026-07-14 11:07:11.664367274 +0000 UTC m=+22.647338109" Jul 14 11:07:12.486129 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.485943 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:12.486282 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:12.486235 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:12.624400 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624372 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624667 2581 generic.go:356] "Generic (PLEG): container finished" podID="a0fa7ed7-bff7-4678-821d-60bd5ea115cd" containerID="1d4e1b890fed1ed5aa73ad6c62497e97eb54c7429a55a52a9b1ad2750f1b856f" exitCode=1 Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624741 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"bd8d3440c559676d848ee3a5ac51d47f985a3f78895cad3b8dc50b35a62a7023"} Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"a594c2af1aa6a234b7cde7f6bde7e20a85cad58dcab961f1dc417f6d28506b16"} Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624848 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"ed4cd783f1a461a543a85219f97213e57998e17715e1090b418ffd9e7a12e3dd"} Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"9acc3e226233d5e1187a8ea7c9e460630368c865d2be4c9fe319b6d3a8a19b68"} Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624912 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerDied","Data":"1d4e1b890fed1ed5aa73ad6c62497e97eb54c7429a55a52a9b1ad2750f1b856f"} Jul 14 11:07:12.625221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.624957 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"6fb4e83fd7f7a87cf28a26bb8132660e557e413cebc03bf0332ecfa56c6f1323"} Jul 14 11:07:12.626103 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.626082 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c" containerID="3a53ca6ff2d8b89a7ef5ab48b85e6720d46c67bb980caf2f003cbc99a463ee57" exitCode=0 Jul 14 11:07:12.626226 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.626191 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerDied","Data":"3a53ca6ff2d8b89a7ef5ab48b85e6720d46c67bb980caf2f003cbc99a463ee57"} Jul 14 11:07:12.640653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.640616 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9rqsm" podStartSLOduration=3.945589209 podStartE2EDuration="23.640603378s" totalImagesPullingTime="19.695014169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:12.640273526 +0000 UTC m=+23.623244362" watchObservedRunningTime="2026-07-14 11:07:12.640603378 +0000 UTC m=+23.623574214" Jul 14 11:07:12.672884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.672791 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9n7wv" podStartSLOduration=5.167214316 podStartE2EDuration="23.672776432s" totalImagesPullingTime="18.505562116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:12.672422927 +0000 UTC m=+23.655393762" watchObservedRunningTime="2026-07-14 11:07:12.672776432 +0000 UTC m=+23.655747267" Jul 14 11:07:12.897158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:12.897132 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 11:07:13.404294 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.404175 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T11:07:12.897151996Z","UUID":"47b4e11c-f8a0-4537-bfb4-36d508f8a49a","Handler":null,"Name":"","Endpoint":""} Jul 14 11:07:13.405850 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.405830 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 11:07:13.405973 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.405871 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 11:07:13.489334 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.489305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:13.489520 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.489306 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:13.489520 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:13.489410 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:13.489520 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:13.489495 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:13.631174 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.630900 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vngvw" event={"ID":"9270024d-9198-42d3-b50f-a38e6d29db56","Type":"ContainerStarted","Data":"63d4f74986db698f096c74f95ce3c1a8643a874e9d0a267901f4677e1b97c14e"} Jul 14 11:07:13.633543 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.633479 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" event={"ID":"14a5cd3f-eae1-4b79-ba37-4ffbf5231281","Type":"ContainerStarted","Data":"b18758ba3a74d7a00a0fba8389e5fd7c5591dcf52f4240add198b181b52a58c6"} Jul 14 11:07:13.644582 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:13.644528 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vngvw" podStartSLOduration=6.142871496 podStartE2EDuration="24.644513954s" totalImagesPullingTime="18.501642458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:13.64415493 +0000 UTC m=+24.627125776" watchObservedRunningTime="2026-07-14 11:07:13.644513954 +0000 UTC m=+24.627484791" Jul 14 11:07:14.486436 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:14.486180 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:14.486582 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:14.486562 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:14.638058 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:14.638020 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" event={"ID":"14a5cd3f-eae1-4b79-ba37-4ffbf5231281","Type":"ContainerStarted","Data":"8618cefc2dce6ad4817a2a2725370345fe7d34ae0c3f01290abcccbe5b86dbf7"} Jul 14 11:07:14.641715 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:14.641693 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:07:14.642167 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:14.642142 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"336f73edd075a8ae563fa095da215681d76638ec739127ba5516c556ca1e8a66"} Jul 14 11:07:14.658913 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:14.658850 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xg5gv" podStartSLOduration=3.8504131790000002 podStartE2EDuration="25.658828094s" totalImagesPullingTime="21.808414915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:14.658402317 +0000 UTC m=+25.641373153" watchObservedRunningTime="2026-07-14 11:07:14.658828094 +0000 UTC m=+25.641798934" Jul 14 11:07:15.486508 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:15.486474 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:15.486703 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:15.486611 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:15.486703 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:15.486666 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:15.486827 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:15.486775 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:16.268814 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:16.268779 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:07:16.269578 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:16.269448 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:07:16.486262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:16.486237 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:16.486441 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:16.486363 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:16.645652 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:16.645628 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:07:16.646076 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:16.646059 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wlfnm" Jul 14 11:07:17.488944 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.488779 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:17.489451 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.488778 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:17.489451 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:17.489014 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:17.489451 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:17.489127 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:17.652953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.652930 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:07:17.653444 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.653422 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"79e78b6d87faa96b3751327b0a544b413cb37b0caf4c07df135bebf70831cffe"} Jul 14 11:07:17.653649 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.653632 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:07:17.653882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.653863 2581 scope.go:117] "RemoveContainer" containerID="1d4e1b890fed1ed5aa73ad6c62497e97eb54c7429a55a52a9b1ad2750f1b856f" Jul 14 11:07:17.655241 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.655221 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c" containerID="86f7b83c3e4379505bb24d4fb01816a829c572749357efbdf364360f5955de85" exitCode=0 Jul 14 11:07:17.655346 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.655299 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerDied","Data":"86f7b83c3e4379505bb24d4fb01816a829c572749357efbdf364360f5955de85"} Jul 14 11:07:17.669611 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:17.669594 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:07:18.486206 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.486177 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:18.486614 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:18.486307 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:18.661521 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.661221 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:07:18.661947 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.661928 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" event={"ID":"a0fa7ed7-bff7-4678-821d-60bd5ea115cd","Type":"ContainerStarted","Data":"756d2f94a1ffd30f6d94d88a20b112d5ce93f4be778b6eacbd04d29ab12704ce"} Jul 14 11:07:18.662155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.662134 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:07:18.662551 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.662415 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:07:18.680655 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.680637 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:07:18.687995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.687958 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" podStartSLOduration=10.883250094 podStartE2EDuration="29.687945127s" totalImagesPullingTime="18.804695033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:18.687076415 +0000 UTC m=+29.670047251" watchObservedRunningTime="2026-07-14 11:07:18.687945127 +0000 UTC m=+29.670915977" Jul 14 11:07:18.703899 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.703874 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mwx66"] Jul 14 11:07:18.704058 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.704033 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:18.704186 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:18.704166 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:18.707141 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.707108 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tp4pg"] Jul 14 11:07:18.707250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.707223 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:18.707348 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:18.707330 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:18.707757 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.707740 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zc6px"] Jul 14 11:07:18.707844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:18.707834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:18.707941 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:18.707923 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:19.666044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:19.666012 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c" containerID="182bd6b65c506222ae05ce1f87a31f60bcb991eb50db89601f9387f737f19c55" exitCode=0 Jul 14 11:07:19.666583 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:19.666100 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerDied","Data":"182bd6b65c506222ae05ce1f87a31f60bcb991eb50db89601f9387f737f19c55"} Jul 14 11:07:20.486528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:20.486497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:20.486700 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:20.486599 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:20.486700 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:20.486605 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:20.486700 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:20.486627 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:20.486819 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:20.486698 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:20.486819 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:20.486765 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:21.672884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:21.672853 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c" containerID="db8d1b318341ec40c244877916590b927fa3be66954bccb2932a508a848ac1e6" exitCode=0 Jul 14 11:07:21.673479 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:21.672909 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerDied","Data":"db8d1b318341ec40c244877916590b927fa3be66954bccb2932a508a848ac1e6"} Jul 14 11:07:22.486097 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:22.486037 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:22.486283 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:22.486161 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:22.486283 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:22.486164 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zc6px" podUID="10f1935a-5778-476c-b896-f6fdc4cfb0d5" Jul 14 11:07:22.486283 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:22.486176 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:22.486283 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:22.486247 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx66" podUID="60660683-0b39-4db6-a3ad-0261e7a7a741" Jul 14 11:07:22.486448 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:22.486335 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tp4pg" podUID="4e1b73a1-619f-41c5-8a62-4380a23919cf" Jul 14 11:07:23.092895 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:23.092860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:23.093332 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.092978 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:07:23.093332 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.093044 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret podName:60660683-0b39-4db6-a3ad-0261e7a7a741 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:55.093030671 +0000 UTC m=+66.076001504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret") pod "global-pull-secret-syncer-mwx66" (UID: "60660683-0b39-4db6-a3ad-0261e7a7a741") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:07:23.194092 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:23.194065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:23.194268 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.194211 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:07:23.194325 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.194279 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:07:55.194260779 +0000 UTC m=+66.177231605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:07:23.294426 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:23.294398 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:23.294574 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.294560 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:07:23.294641 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.294583 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:07:23.294641 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.294596 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dl8dx for pod openshift-network-diagnostics/network-check-target-zc6px: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:07:23.294771 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:23.294653 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx podName:10f1935a-5778-476c-b896-f6fdc4cfb0d5 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:55.294635261 +0000 UTC m=+66.277606081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dl8dx" (UniqueName: "kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx") pod "network-check-target-zc6px" (UID: "10f1935a-5778-476c-b896-f6fdc4cfb0d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:07:24.375469 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.375440 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeReady" Jul 14 11:07:24.375851 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.375577 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 11:07:24.418471 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.418443 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c7pwn"] Jul 14 11:07:24.421061 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.421036 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.421376 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.421353 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fqw6w"] Jul 14 11:07:24.423249 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.423059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.425827 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.425807 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 11:07:24.425929 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.425835 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-grcgd\"" Jul 14 11:07:24.425929 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.425804 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 11:07:24.426191 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.426172 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 11:07:24.426481 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.426463 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 11:07:24.426651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.426633 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7vlft\"" Jul 14 11:07:24.426651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.426645 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 11:07:24.431701 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.431681 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c7pwn"] Jul 14 11:07:24.439372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.439350 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fqw6w"] Jul 14 11:07:24.485784 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.485758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:24.485784 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.485766 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:24.486346 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.486141 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:24.487701 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.487683 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 11:07:24.487834 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.487683 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zjjh4\"" Jul 14 11:07:24.488061 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.488039 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 11:07:24.488177 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.488062 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 11:07:24.488177 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.488046 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 11:07:24.488292 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.488227 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhbfh\"" Jul 14 11:07:24.501907 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.501889 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08a7f632-942c-4b96-893e-ce794f5b1932-config-volume\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.502011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.501971 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.502011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.502003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/e28243da-e892-4689-8556-41b7e019f242-kube-api-access-9957t\") pod \"ingress-canary-fqw6w\" (UID: \"e28243da-e892-4689-8556-41b7e019f242\") " pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.502140 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.502028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg7zd\" (UniqueName: \"kubernetes.io/projected/08a7f632-942c-4b96-893e-ce794f5b1932-kube-api-access-jg7zd\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.502140 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.502070 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28243da-e892-4689-8556-41b7e019f242-cert\") pod \"ingress-canary-fqw6w\" (UID: \"e28243da-e892-4689-8556-41b7e019f242\") " pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.502218 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.502106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a7f632-942c-4b96-893e-ce794f5b1932-tmp-dir\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.603309 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.603271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.603459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.603331 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/e28243da-e892-4689-8556-41b7e019f242-kube-api-access-9957t\") pod \"ingress-canary-fqw6w\" (UID: \"e28243da-e892-4689-8556-41b7e019f242\") " pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.603459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.603360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg7zd\" (UniqueName: \"kubernetes.io/projected/08a7f632-942c-4b96-893e-ce794f5b1932-kube-api-access-jg7zd\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.603459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.603381 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28243da-e892-4689-8556-41b7e019f242-cert\") pod \"ingress-canary-fqw6w\" (UID: \"e28243da-e892-4689-8556-41b7e019f242\") " pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.603459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.603414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a7f632-942c-4b96-893e-ce794f5b1932-tmp-dir\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.603459 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:24.603431 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:07:24.603459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.603437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08a7f632-942c-4b96-893e-ce794f5b1932-config-volume\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.603772 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:24.603499 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls podName:08a7f632-942c-4b96-893e-ce794f5b1932 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:25.103481459 +0000 UTC m=+36.086452287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls") pod "dns-default-c7pwn" (UID: "08a7f632-942c-4b96-893e-ce794f5b1932") : secret "dns-default-metrics-tls" not found Jul 14 11:07:24.604429 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.604376 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a7f632-942c-4b96-893e-ce794f5b1932-tmp-dir\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.604566 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.604545 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08a7f632-942c-4b96-893e-ce794f5b1932-config-volume\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.609027 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.609004 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e28243da-e892-4689-8556-41b7e019f242-cert\") pod \"ingress-canary-fqw6w\" (UID: \"e28243da-e892-4689-8556-41b7e019f242\") " pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.611938 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.611918 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg7zd\" (UniqueName: \"kubernetes.io/projected/08a7f632-942c-4b96-893e-ce794f5b1932-kube-api-access-jg7zd\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:24.612064 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.612043 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9957t\" (UniqueName: \"kubernetes.io/projected/e28243da-e892-4689-8556-41b7e019f242-kube-api-access-9957t\") pod \"ingress-canary-fqw6w\" (UID: \"e28243da-e892-4689-8556-41b7e019f242\") " pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:24.740162 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:24.740057 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fqw6w" Jul 14 11:07:25.107385 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:25.107306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:25.107537 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:25.107485 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:07:25.107587 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:25.107549 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls podName:08a7f632-942c-4b96-893e-ce794f5b1932 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:26.107532078 +0000 UTC m=+37.090502909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls") pod "dns-default-c7pwn" (UID: "08a7f632-942c-4b96-893e-ce794f5b1932") : secret "dns-default-metrics-tls" not found Jul 14 11:07:25.708524 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:25.708496 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fqw6w"] Jul 14 11:07:25.712552 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:25.712526 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28243da_e892_4689_8556_41b7e019f242.slice/crio-df28dc0063e755025dbe320dfbf1a2dfd0781d72573c9bdec71e4dfaf46e39f9 WatchSource:0}: Error finding container df28dc0063e755025dbe320dfbf1a2dfd0781d72573c9bdec71e4dfaf46e39f9: Status 404 returned error can't find the container with id df28dc0063e755025dbe320dfbf1a2dfd0781d72573c9bdec71e4dfaf46e39f9 Jul 14 11:07:26.114083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:26.114054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:26.114237 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:26.114180 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:07:26.114284 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:26.114245 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls podName:08a7f632-942c-4b96-893e-ce794f5b1932 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:28.114226369 +0000 UTC m=+39.097197188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls") pod "dns-default-c7pwn" (UID: "08a7f632-942c-4b96-893e-ce794f5b1932") : secret "dns-default-metrics-tls" not found Jul 14 11:07:26.689716 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:26.689680 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c" containerID="3a3f10076f7dc440564c9d2f68bec38b0ef5cd4d9522c2ad7c8621c103db3b0e" exitCode=0 Jul 14 11:07:26.689897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:26.689762 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerDied","Data":"3a3f10076f7dc440564c9d2f68bec38b0ef5cd4d9522c2ad7c8621c103db3b0e"} Jul 14 11:07:26.691046 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:26.690997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fqw6w" event={"ID":"e28243da-e892-4689-8556-41b7e019f242","Type":"ContainerStarted","Data":"df28dc0063e755025dbe320dfbf1a2dfd0781d72573c9bdec71e4dfaf46e39f9"} Jul 14 11:07:27.698017 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:27.697830 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c" containerID="ee7f29ed4e9186545b2b9fe841987ecd37e6144720377eebb88eda41d886a3e8" exitCode=0 Jul 14 11:07:27.698413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:27.697872 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerDied","Data":"ee7f29ed4e9186545b2b9fe841987ecd37e6144720377eebb88eda41d886a3e8"} Jul 14 11:07:28.128995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:28.128971 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:28.129092 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:28.129066 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:07:28.129144 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:28.129134 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls podName:08a7f632-942c-4b96-893e-ce794f5b1932 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:32.129103909 +0000 UTC m=+43.112074721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls") pod "dns-default-c7pwn" (UID: "08a7f632-942c-4b96-893e-ce794f5b1932") : secret "dns-default-metrics-tls" not found Jul 14 11:07:28.706638 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:28.706593 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j2466" event={"ID":"f8e8dd76-65c8-4bea-adb5-c5a4cae0e93c","Type":"ContainerStarted","Data":"5298fc84bb83681580d50fbcf798b22e13f2c1f75ffbcfd7ddfdf2b4646cf839"} Jul 14 11:07:28.708149 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:28.708110 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fqw6w" event={"ID":"e28243da-e892-4689-8556-41b7e019f242","Type":"ContainerStarted","Data":"aff28ce0608c93b6e23907d6a10047e66401985f15d7efebae363a873c1a1db5"} Jul 14 11:07:28.727386 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:28.727348 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j2466" podStartSLOduration=10.855807577 podStartE2EDuration="39.72733539s" totalImagesPullingTime="28.871527813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:28.726145436 +0000 UTC m=+39.709116272" watchObservedRunningTime="2026-07-14 11:07:28.72733539 +0000 UTC m=+39.710306225" Jul 14 11:07:28.740726 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:28.740695 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fqw6w" podStartSLOduration=2.138823592 podStartE2EDuration="4.740683654s" totalImagesPullingTime="2.601860062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:28.739725409 +0000 UTC m=+39.722696244" watchObservedRunningTime="2026-07-14 11:07:28.740683654 +0000 UTC m=+39.723654488" Jul 14 11:07:32.156763 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:32.156729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:32.157275 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:32.156834 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:07:32.157275 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:32.156891 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls podName:08a7f632-942c-4b96-893e-ce794f5b1932 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:40.156879786 +0000 UTC m=+51.139850598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls") pod "dns-default-c7pwn" (UID: "08a7f632-942c-4b96-893e-ce794f5b1932") : secret "dns-default-metrics-tls" not found Jul 14 11:07:33.342997 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.342965 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-hrpl5"] Jul 14 11:07:33.372623 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.372602 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-hrpl5"] Jul 14 11:07:33.372789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.372684 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.374609 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.374583 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qzr2p\"" Jul 14 11:07:33.374739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.374720 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 14 11:07:33.375053 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.375030 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 14 11:07:33.375184 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.375082 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 14 11:07:33.375184 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.375033 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:07:33.379953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.379933 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 14 11:07:33.438294 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.438273 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2"] Jul 14 11:07:33.456565 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.456548 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg"] Jul 14 11:07:33.456670 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.456655 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" Jul 14 11:07:33.458666 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.458647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-flbxc\"" Jul 14 11:07:33.466092 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.466074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sc4n\" (UniqueName: \"kubernetes.io/projected/bca091db-f43d-447d-beb1-24cb429d4aad-kube-api-access-8sc4n\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.466192 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.466106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bca091db-f43d-447d-beb1-24cb429d4aad-config\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.466232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.466218 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bca091db-f43d-447d-beb1-24cb429d4aad-serving-cert\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.466267 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.466243 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bca091db-f43d-447d-beb1-24cb429d4aad-trusted-ca\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.482732 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.482707 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g"] Jul 14 11:07:33.482830 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.482817 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.484980 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.484963 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 11:07:33.485203 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.485189 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 11:07:33.485421 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.485404 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:07:33.485467 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.485421 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 11:07:33.485467 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.485458 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-twjgd\"" Jul 14 11:07:33.500635 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.500619 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.502362 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.502344 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 14 11:07:33.502675 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.502654 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9tdmz\"" Jul 14 11:07:33.502768 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.502744 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 11:07:33.503064 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.503034 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 14 11:07:33.503164 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.503108 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 11:07:33.504003 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.503988 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2"] Jul 14 11:07:33.504071 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.504017 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg"] Jul 14 11:07:33.504071 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.504032 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g"] Jul 14 11:07:33.567031 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bca091db-f43d-447d-beb1-24cb429d4aad-config\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.567110 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf276\" (UniqueName: \"kubernetes.io/projected/5cd8c426-32f9-4cf1-b3e7-cfb239c209fe-kube-api-access-hf276\") pod \"network-check-source-869dcfcdcd-swtn2\" (UID: \"5cd8c426-32f9-4cf1-b3e7-cfb239c209fe\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" Jul 14 11:07:33.567110 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d63cc1-acc6-435e-8f08-70a84ed537a5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.567226 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567207 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bca091db-f43d-447d-beb1-24cb429d4aad-serving-cert\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.567261 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567236 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bca091db-f43d-447d-beb1-24cb429d4aad-trusted-ca\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.567299 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgrnc\" (UniqueName: \"kubernetes.io/projected/f9d63cc1-acc6-435e-8f08-70a84ed537a5-kube-api-access-rgrnc\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.567334 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sc4n\" (UniqueName: \"kubernetes.io/projected/bca091db-f43d-447d-beb1-24cb429d4aad-kube-api-access-8sc4n\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.567379 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567345 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9d63cc1-acc6-435e-8f08-70a84ed537a5-config\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.567659 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567644 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bca091db-f43d-447d-beb1-24cb429d4aad-config\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.567981 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.567957 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bca091db-f43d-447d-beb1-24cb429d4aad-trusted-ca\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.570619 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.570602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bca091db-f43d-447d-beb1-24cb429d4aad-serving-cert\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.576940 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.576921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sc4n\" (UniqueName: \"kubernetes.io/projected/bca091db-f43d-447d-beb1-24cb429d4aad-kube-api-access-8sc4n\") pod \"console-operator-7866497494-hrpl5\" (UID: \"bca091db-f43d-447d-beb1-24cb429d4aad\") " pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.668039 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrnc\" (UniqueName: \"kubernetes.io/projected/f9d63cc1-acc6-435e-8f08-70a84ed537a5-kube-api-access-rgrnc\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.668159 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9d63cc1-acc6-435e-8f08-70a84ed537a5-config\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.668159 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj95s\" (UniqueName: \"kubernetes.io/projected/6986c71b-b111-413a-95e7-30a00b1d2068-kube-api-access-lj95s\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.668159 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6986c71b-b111-413a-95e7-30a00b1d2068-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.668278 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf276\" (UniqueName: \"kubernetes.io/projected/5cd8c426-32f9-4cf1-b3e7-cfb239c209fe-kube-api-access-hf276\") pod \"network-check-source-869dcfcdcd-swtn2\" (UID: \"5cd8c426-32f9-4cf1-b3e7-cfb239c209fe\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" Jul 14 11:07:33.668278 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d63cc1-acc6-435e-8f08-70a84ed537a5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.668278 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668240 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.668604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.668587 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9d63cc1-acc6-435e-8f08-70a84ed537a5-config\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.670058 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.670041 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d63cc1-acc6-435e-8f08-70a84ed537a5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.675478 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.675455 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgrnc\" (UniqueName: \"kubernetes.io/projected/f9d63cc1-acc6-435e-8f08-70a84ed537a5-kube-api-access-rgrnc\") pod \"service-ca-operator-7695fc6c8c-9w9hg\" (UID: \"f9d63cc1-acc6-435e-8f08-70a84ed537a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.675735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.675719 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf276\" (UniqueName: \"kubernetes.io/projected/5cd8c426-32f9-4cf1-b3e7-cfb239c209fe-kube-api-access-hf276\") pod \"network-check-source-869dcfcdcd-swtn2\" (UID: \"5cd8c426-32f9-4cf1-b3e7-cfb239c209fe\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" Jul 14 11:07:33.682506 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.682487 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:33.766961 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.766790 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" Jul 14 11:07:33.769093 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.769062 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.769228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.769175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj95s\" (UniqueName: \"kubernetes.io/projected/6986c71b-b111-413a-95e7-30a00b1d2068-kube-api-access-lj95s\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.769285 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.769238 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6986c71b-b111-413a-95e7-30a00b1d2068-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.769285 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:33.769255 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:33.769385 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:33.769343 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls podName:6986c71b-b111-413a-95e7-30a00b1d2068 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:34.269320604 +0000 UTC m=+45.252291418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cf46g" (UID: "6986c71b-b111-413a-95e7-30a00b1d2068") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:33.770031 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.769984 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6986c71b-b111-413a-95e7-30a00b1d2068-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.777359 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.777340 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj95s\" (UniqueName: \"kubernetes.io/projected/6986c71b-b111-413a-95e7-30a00b1d2068-kube-api-access-lj95s\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:33.791586 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.791568 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" Jul 14 11:07:33.828416 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.828389 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-hrpl5"] Jul 14 11:07:33.832228 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:33.832200 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca091db_f43d_447d_beb1_24cb429d4aad.slice/crio-1d0a3d3351cf3cb549e10e5688cac023fbd86fc18f10c21bf1e61fbb69f4153d WatchSource:0}: Error finding container 1d0a3d3351cf3cb549e10e5688cac023fbd86fc18f10c21bf1e61fbb69f4153d: Status 404 returned error can't find the container with id 1d0a3d3351cf3cb549e10e5688cac023fbd86fc18f10c21bf1e61fbb69f4153d Jul 14 11:07:33.937612 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.937400 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2"] Jul 14 11:07:33.942201 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:33.941933 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd8c426_32f9_4cf1_b3e7_cfb239c209fe.slice/crio-5585cac9f43ae1fbaa562c6a4d34c12fd333c87d27a46b6f913464e8967873fb WatchSource:0}: Error finding container 5585cac9f43ae1fbaa562c6a4d34c12fd333c87d27a46b6f913464e8967873fb: Status 404 returned error can't find the container with id 5585cac9f43ae1fbaa562c6a4d34c12fd333c87d27a46b6f913464e8967873fb Jul 14 11:07:33.978319 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:33.978294 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg"] Jul 14 11:07:33.981353 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:33.981328 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d63cc1_acc6_435e_8f08_70a84ed537a5.slice/crio-39eb4f4e9efa61364465b2e8767132dd14d5432fb12b439ac6d8c21a560afe42 WatchSource:0}: Error finding container 39eb4f4e9efa61364465b2e8767132dd14d5432fb12b439ac6d8c21a560afe42: Status 404 returned error can't find the container with id 39eb4f4e9efa61364465b2e8767132dd14d5432fb12b439ac6d8c21a560afe42 Jul 14 11:07:34.273826 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:34.273749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:34.273946 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:34.273899 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:34.273988 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:34.273978 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls podName:6986c71b-b111-413a-95e7-30a00b1d2068 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:35.273961437 +0000 UTC m=+46.256932250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cf46g" (UID: "6986c71b-b111-413a-95e7-30a00b1d2068") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:34.723168 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:34.723109 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hrpl5" event={"ID":"bca091db-f43d-447d-beb1-24cb429d4aad","Type":"ContainerStarted","Data":"1d0a3d3351cf3cb549e10e5688cac023fbd86fc18f10c21bf1e61fbb69f4153d"} Jul 14 11:07:34.724556 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:34.724521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" event={"ID":"f9d63cc1-acc6-435e-8f08-70a84ed537a5","Type":"ContainerStarted","Data":"39eb4f4e9efa61364465b2e8767132dd14d5432fb12b439ac6d8c21a560afe42"} Jul 14 11:07:34.725742 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:34.725718 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" event={"ID":"5cd8c426-32f9-4cf1-b3e7-cfb239c209fe","Type":"ContainerStarted","Data":"5585cac9f43ae1fbaa562c6a4d34c12fd333c87d27a46b6f913464e8967873fb"} Jul 14 11:07:35.282690 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:35.282285 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:35.282690 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:35.282454 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:35.282690 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:35.282515 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls podName:6986c71b-b111-413a-95e7-30a00b1d2068 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:37.282496947 +0000 UTC m=+48.265467765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cf46g" (UID: "6986c71b-b111-413a-95e7-30a00b1d2068") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:37.298949 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.298904 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:37.299420 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:37.299080 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:37.299420 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:37.299201 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls podName:6986c71b-b111-413a-95e7-30a00b1d2068 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.299176707 +0000 UTC m=+52.282147535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cf46g" (UID: "6986c71b-b111-413a-95e7-30a00b1d2068") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:37.426579 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.426551 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7sf5x"] Jul 14 11:07:37.459972 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.459933 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7sf5x"] Jul 14 11:07:37.460130 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.460055 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.462090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.462061 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 11:07:37.462227 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.462175 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 11:07:37.462431 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.462326 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 11:07:37.462546 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.462526 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-24zbp\"" Jul 14 11:07:37.462773 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.462746 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 11:07:37.602483 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.602400 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mxmg\" (UniqueName: \"kubernetes.io/projected/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-api-access-7mxmg\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.602483 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.602459 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.602695 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.602537 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/091c0e73-d9d2-4892-8912-3b249e1dfcbb-crio-socket\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.602695 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.602592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.602695 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.602665 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/091c0e73-d9d2-4892-8912-3b249e1dfcbb-data-volume\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.703615 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.703591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mxmg\" (UniqueName: \"kubernetes.io/projected/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-api-access-7mxmg\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.703777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.703641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.703777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.703666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/091c0e73-d9d2-4892-8912-3b249e1dfcbb-crio-socket\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.703777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.703686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.703777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.703713 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/091c0e73-d9d2-4892-8912-3b249e1dfcbb-data-volume\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.703966 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:37.703817 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:07:37.703966 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:37.703839 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:07:37.703966 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:37.703878 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:38.203859472 +0000 UTC m=+49.186830297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : secret "insights-runtime-extractor-tls" not found Jul 14 11:07:37.703966 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:37.703916 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:38.203898091 +0000 UTC m=+49.186868907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : configmap "kube-rbac-proxy" not found Jul 14 11:07:37.703966 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.703913 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/091c0e73-d9d2-4892-8912-3b249e1dfcbb-crio-socket\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.704239 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.704021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/091c0e73-d9d2-4892-8912-3b249e1dfcbb-data-volume\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:37.714431 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:37.714411 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mxmg\" (UniqueName: \"kubernetes.io/projected/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-api-access-7mxmg\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:38.207843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.207813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:38.207955 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.207858 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:38.207955 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:38.207927 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:07:38.208020 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:38.207954 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:07:38.208020 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:38.207980 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:39.207966114 +0000 UTC m=+50.190936927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : configmap "kube-rbac-proxy" not found Jul 14 11:07:38.208020 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:38.207996 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:39.207985041 +0000 UTC m=+50.190955858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : secret "insights-runtime-extractor-tls" not found Jul 14 11:07:38.738265 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.738186 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/0.log" Jul 14 11:07:38.738265 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.738238 2581 generic.go:356] "Generic (PLEG): container finished" podID="bca091db-f43d-447d-beb1-24cb429d4aad" containerID="d3aadaf3f3c7f1aceb41b27d3469bd87445114697326da738f778ebd1653444b" exitCode=255 Jul 14 11:07:38.738747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.738289 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hrpl5" event={"ID":"bca091db-f43d-447d-beb1-24cb429d4aad","Type":"ContainerDied","Data":"d3aadaf3f3c7f1aceb41b27d3469bd87445114697326da738f778ebd1653444b"} Jul 14 11:07:38.738747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.738601 2581 scope.go:117] "RemoveContainer" containerID="d3aadaf3f3c7f1aceb41b27d3469bd87445114697326da738f778ebd1653444b" Jul 14 11:07:38.739970 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.739942 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" event={"ID":"f9d63cc1-acc6-435e-8f08-70a84ed537a5","Type":"ContainerStarted","Data":"627d435ad2fdec9a631658e2e91475f63daa0e2fd7c97f0de28aced0ccd0fd8d"} Jul 14 11:07:38.741526 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.741507 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" event={"ID":"5cd8c426-32f9-4cf1-b3e7-cfb239c209fe","Type":"ContainerStarted","Data":"24d8011aadc2826220f9d788daf464461e106d971836bf8c8b4d6534e4ac11fd"} Jul 14 11:07:38.783054 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.782225 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" podStartSLOduration=1.410597488 podStartE2EDuration="5.782206802s" totalImagesPullingTime="4.371609314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:38.781771105 +0000 UTC m=+49.764741942" watchObservedRunningTime="2026-07-14 11:07:38.782206802 +0000 UTC m=+49.765177639" Jul 14 11:07:38.800067 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:38.800021 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-swtn2" podStartSLOduration=1.381154134 podStartE2EDuration="5.800004012s" totalImagesPullingTime="4.418849878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:38.799510791 +0000 UTC m=+49.782481627" watchObservedRunningTime="2026-07-14 11:07:38.800004012 +0000 UTC m=+49.782974849" Jul 14 11:07:39.217131 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.217086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:39.217303 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.217150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:39.217303 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:39.217213 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:07:39.217303 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:39.217264 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:07:39.217303 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:39.217285 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.217266326 +0000 UTC m=+52.200237153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : configmap "kube-rbac-proxy" not found Jul 14 11:07:39.217303 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:39.217303 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:41.217292743 +0000 UTC m=+52.200263556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : secret "insights-runtime-extractor-tls" not found Jul 14 11:07:39.268127 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.268089 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9rqsm_8e708d4a-7e79-4aae-b247-fbdea80ca0f5/dns-node-resolver/0.log" Jul 14 11:07:39.745459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.745435 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:07:39.745845 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.745817 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/0.log" Jul 14 11:07:39.745887 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.745849 2581 generic.go:356] "Generic (PLEG): container finished" podID="bca091db-f43d-447d-beb1-24cb429d4aad" containerID="6fdcbf189f9b8d6e7077662f37d9d3d746707309c578d2a26089047c05e2fc0e" exitCode=255 Jul 14 11:07:39.745957 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.745935 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hrpl5" event={"ID":"bca091db-f43d-447d-beb1-24cb429d4aad","Type":"ContainerDied","Data":"6fdcbf189f9b8d6e7077662f37d9d3d746707309c578d2a26089047c05e2fc0e"} Jul 14 11:07:39.746004 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.745993 2581 scope.go:117] "RemoveContainer" containerID="d3aadaf3f3c7f1aceb41b27d3469bd87445114697326da738f778ebd1653444b" Jul 14 11:07:39.746275 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:39.746253 2581 scope.go:117] "RemoveContainer" containerID="6fdcbf189f9b8d6e7077662f37d9d3d746707309c578d2a26089047c05e2fc0e" Jul 14 11:07:39.746429 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:39.746410 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-hrpl5_openshift-console-operator(bca091db-f43d-447d-beb1-24cb429d4aad)\"" pod="openshift-console-operator/console-operator-7866497494-hrpl5" podUID="bca091db-f43d-447d-beb1-24cb429d4aad" Jul 14 11:07:40.069750 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:40.069686 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9n7wv_8eb6e134-f251-4b16-8c61-9a8e76e30d4d/node-ca/0.log" Jul 14 11:07:40.224867 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:40.224839 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:40.225023 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:40.224951 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:07:40.225023 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:40.225002 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls podName:08a7f632-942c-4b96-893e-ce794f5b1932 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:56.224989231 +0000 UTC m=+67.207960044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls") pod "dns-default-c7pwn" (UID: "08a7f632-942c-4b96-893e-ce794f5b1932") : secret "dns-default-metrics-tls" not found Jul 14 11:07:40.749322 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:40.749299 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:07:40.749780 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:40.749648 2581 scope.go:117] "RemoveContainer" containerID="6fdcbf189f9b8d6e7077662f37d9d3d746707309c578d2a26089047c05e2fc0e" Jul 14 11:07:40.749854 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:40.749828 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-hrpl5_openshift-console-operator(bca091db-f43d-447d-beb1-24cb429d4aad)\"" pod="openshift-console-operator/console-operator-7866497494-hrpl5" podUID="bca091db-f43d-447d-beb1-24cb429d4aad" Jul 14 11:07:40.867304 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:40.867276 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fqw6w_e28243da-e892-4689-8556-41b7e019f242/serve-healthcheck-canary/0.log" Jul 14 11:07:41.231944 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:41.231915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:41.232104 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:41.232035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:41.232104 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:41.232060 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:07:41.232104 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:41.232102 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:07:41.232246 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:41.232144 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:45.232109315 +0000 UTC m=+56.215080128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : secret "insights-runtime-extractor-tls" not found Jul 14 11:07:41.232246 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:41.232161 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:45.232154164 +0000 UTC m=+56.215124978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : configmap "kube-rbac-proxy" not found Jul 14 11:07:41.333139 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:41.333098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:41.333294 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:41.333254 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:41.333422 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:41.333324 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls podName:6986c71b-b111-413a-95e7-30a00b1d2068 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:49.333303726 +0000 UTC m=+60.316274542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cf46g" (UID: "6986c71b-b111-413a-95e7-30a00b1d2068") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:42.350908 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.350879 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8657c45967-npz6g"] Jul 14 11:07:42.373061 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.373037 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8657c45967-npz6g"] Jul 14 11:07:42.373193 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.373149 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.375212 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.375189 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 11:07:42.375330 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.375268 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 11:07:42.375417 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.375402 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cn4jk\"" Jul 14 11:07:42.375470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.375456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 11:07:42.382083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.382065 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 11:07:42.439539 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439513 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-installation-pull-secrets\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439619 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-bound-sa-token\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439658 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439694 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439673 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdmdd\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-kube-api-access-kdmdd\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439731 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-trusted-ca\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439731 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e49fe4-ed80-4c3a-b73a-feb001215541-ca-trust-extracted\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439791 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-certificates\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.439823 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.439793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-image-registry-private-configuration\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540318 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540290 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e49fe4-ed80-4c3a-b73a-feb001215541-ca-trust-extracted\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540399 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-certificates\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540399 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-image-registry-private-configuration\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540587 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-installation-pull-secrets\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540625 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-bound-sa-token\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540713 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540683 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e49fe4-ed80-4c3a-b73a-feb001215541-ca-trust-extracted\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540713 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540696 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540901 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdmdd\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-kube-api-access-kdmdd\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540901 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540769 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-trusted-ca\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.540901 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:42.540860 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:07:42.540901 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:42.540878 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8657c45967-npz6g: secret "image-registry-tls" not found Jul 14 11:07:42.541170 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:42.540940 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls podName:b6e49fe4-ed80-4c3a-b73a-feb001215541 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:43.040922489 +0000 UTC m=+54.023893311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls") pod "image-registry-8657c45967-npz6g" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541") : secret "image-registry-tls" not found Jul 14 11:07:42.541170 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.540949 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-certificates\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.541665 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.541643 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-trusted-ca\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.542756 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.542739 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-image-registry-private-configuration\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.542875 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.542859 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-installation-pull-secrets\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.550858 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.550839 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdmdd\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-kube-api-access-kdmdd\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:42.550858 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:42.550855 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-bound-sa-token\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:43.044460 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:43.044430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:43.044669 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:43.044537 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:07:43.044669 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:43.044548 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8657c45967-npz6g: secret "image-registry-tls" not found Jul 14 11:07:43.044669 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:43.044594 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls podName:b6e49fe4-ed80-4c3a-b73a-feb001215541 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:44.044582254 +0000 UTC m=+55.027553066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls") pod "image-registry-8657c45967-npz6g" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541") : secret "image-registry-tls" not found Jul 14 11:07:43.682957 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:43.682926 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:43.683419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:43.682979 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:43.683419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:43.683377 2581 scope.go:117] "RemoveContainer" containerID="6fdcbf189f9b8d6e7077662f37d9d3d746707309c578d2a26089047c05e2fc0e" Jul 14 11:07:43.683576 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:43.683554 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-hrpl5_openshift-console-operator(bca091db-f43d-447d-beb1-24cb429d4aad)\"" pod="openshift-console-operator/console-operator-7866497494-hrpl5" podUID="bca091db-f43d-447d-beb1-24cb429d4aad" Jul 14 11:07:44.051497 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:44.051425 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:44.051632 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:44.051530 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:07:44.051632 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:44.051541 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8657c45967-npz6g: secret "image-registry-tls" not found Jul 14 11:07:44.051632 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:44.051586 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls podName:b6e49fe4-ed80-4c3a-b73a-feb001215541 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:46.051574038 +0000 UTC m=+57.034544851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls") pod "image-registry-8657c45967-npz6g" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541") : secret "image-registry-tls" not found Jul 14 11:07:45.259378 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:45.259341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:45.259822 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:45.259385 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:45.259822 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:45.259461 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:07:45.259822 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:45.259495 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:07:45.259822 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:45.259524 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:53.259509104 +0000 UTC m=+64.242479917 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : configmap "kube-rbac-proxy" not found Jul 14 11:07:45.259822 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:45.259541 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:07:53.259530515 +0000 UTC m=+64.242501328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : secret "insights-runtime-extractor-tls" not found Jul 14 11:07:46.065060 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:46.065028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:46.065239 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:46.065194 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:07:46.065239 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:46.065211 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8657c45967-npz6g: secret "image-registry-tls" not found Jul 14 11:07:46.065310 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:46.065261 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls podName:b6e49fe4-ed80-4c3a-b73a-feb001215541 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:50.065247061 +0000 UTC m=+61.048217874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls") pod "image-registry-8657c45967-npz6g" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541") : secret "image-registry-tls" not found Jul 14 11:07:49.389496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:49.389464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:07:49.389866 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:49.389601 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:49.389866 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:49.389653 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls podName:6986c71b-b111-413a-95e7-30a00b1d2068 nodeName:}" failed. No retries permitted until 2026-07-14 11:08:05.389639531 +0000 UTC m=+76.372610344 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cf46g" (UID: "6986c71b-b111-413a-95e7-30a00b1d2068") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:07:50.094375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:50.094336 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:50.094571 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:50.094466 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:07:50.094571 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:50.094482 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8657c45967-npz6g: secret "image-registry-tls" not found Jul 14 11:07:50.094571 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:50.094551 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls podName:b6e49fe4-ed80-4c3a-b73a-feb001215541 nodeName:}" failed. No retries permitted until 2026-07-14 11:07:58.094534324 +0000 UTC m=+69.077505146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls") pod "image-registry-8657c45967-npz6g" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541") : secret "image-registry-tls" not found Jul 14 11:07:50.687696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:50.687670 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2w2v" Jul 14 11:07:53.318707 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:53.318673 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:53.318707 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:53.318710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:53.319107 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:53.318846 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:07:53.319107 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:53.318895 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls podName:091c0e73-d9d2-4892-8912-3b249e1dfcbb nodeName:}" failed. No retries permitted until 2026-07-14 11:08:09.318882583 +0000 UTC m=+80.301853396 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7sf5x" (UID: "091c0e73-d9d2-4892-8912-3b249e1dfcbb") : secret "insights-runtime-extractor-tls" not found Jul 14 11:07:53.319225 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:53.319157 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/091c0e73-d9d2-4892-8912-3b249e1dfcbb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:07:55.130997 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.130953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:55.133489 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.133468 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 11:07:55.143379 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.143353 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60660683-0b39-4db6-a3ad-0261e7a7a741-original-pull-secret\") pod \"global-pull-secret-syncer-mwx66\" (UID: \"60660683-0b39-4db6-a3ad-0261e7a7a741\") " pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:55.231698 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.231679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:07:55.233612 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.233593 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 11:07:55.241990 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:55.241971 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 11:07:55.242048 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:07:55.242019 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs podName:4e1b73a1-619f-41c5-8a62-4380a23919cf nodeName:}" failed. No retries permitted until 2026-07-14 11:08:59.242005912 +0000 UTC m=+130.224976725 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs") pod "network-metrics-daemon-tp4pg" (UID: "4e1b73a1-619f-41c5-8a62-4380a23919cf") : secret "metrics-daemon-secret" not found Jul 14 11:07:55.332430 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.332407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:55.334670 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.334655 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl8dx\" (UniqueName: \"kubernetes.io/projected/10f1935a-5778-476c-b896-f6fdc4cfb0d5-kube-api-access-dl8dx\") pod \"network-check-target-zc6px\" (UID: \"10f1935a-5778-476c-b896-f6fdc4cfb0d5\") " pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:55.397394 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.397343 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zjjh4\"" Jul 14 11:07:55.402764 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.402752 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx66" Jul 14 11:07:55.405562 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.405545 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:55.529490 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.529466 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zc6px"] Jul 14 11:07:55.532389 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:55.532364 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f1935a_5778_476c_b896_f6fdc4cfb0d5.slice/crio-b6bb7bfccecf8382acdaa70e8d6ca1c946eb0e92548a24ef87ef11c68a86d200 WatchSource:0}: Error finding container b6bb7bfccecf8382acdaa70e8d6ca1c946eb0e92548a24ef87ef11c68a86d200: Status 404 returned error can't find the container with id b6bb7bfccecf8382acdaa70e8d6ca1c946eb0e92548a24ef87ef11c68a86d200 Jul 14 11:07:55.545150 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.545097 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mwx66"] Jul 14 11:07:55.549722 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:55.549697 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60660683_0b39_4db6_a3ad_0261e7a7a741.slice/crio-f529b6b95d5fc2fe9c94619abe72c5dfd0c3f5170dbccb108e96df340ad1b158 WatchSource:0}: Error finding container f529b6b95d5fc2fe9c94619abe72c5dfd0c3f5170dbccb108e96df340ad1b158: Status 404 returned error can't find the container with id f529b6b95d5fc2fe9c94619abe72c5dfd0c3f5170dbccb108e96df340ad1b158 Jul 14 11:07:55.788169 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.788077 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zc6px" event={"ID":"10f1935a-5778-476c-b896-f6fdc4cfb0d5","Type":"ContainerStarted","Data":"2ad7842cc75dffa6a5b7a28a66e3ffdc8a349e6aa6a06e77b4ca1be624079034"} Jul 14 11:07:55.788169 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.788154 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:07:55.788350 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.788177 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zc6px" event={"ID":"10f1935a-5778-476c-b896-f6fdc4cfb0d5","Type":"ContainerStarted","Data":"b6bb7bfccecf8382acdaa70e8d6ca1c946eb0e92548a24ef87ef11c68a86d200"} Jul 14 11:07:55.789160 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.789101 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mwx66" event={"ID":"60660683-0b39-4db6-a3ad-0261e7a7a741","Type":"ContainerStarted","Data":"f529b6b95d5fc2fe9c94619abe72c5dfd0c3f5170dbccb108e96df340ad1b158"} Jul 14 11:07:55.803479 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:55.803439 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zc6px" podStartSLOduration=66.803426233 podStartE2EDuration="1m6.803426233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:55.802845313 +0000 UTC m=+66.785816145" watchObservedRunningTime="2026-07-14 11:07:55.803426233 +0000 UTC m=+66.786397066" Jul 14 11:07:56.240189 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:56.240157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:56.242726 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:56.242703 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a7f632-942c-4b96-893e-ce794f5b1932-metrics-tls\") pod \"dns-default-c7pwn\" (UID: \"08a7f632-942c-4b96-893e-ce794f5b1932\") " pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:56.536306 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:56.536233 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-grcgd\"" Jul 14 11:07:56.544836 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:56.544813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c7pwn" Jul 14 11:07:56.683639 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:56.683602 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c7pwn"] Jul 14 11:07:56.687935 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:07:56.687902 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a7f632_942c_4b96_893e_ce794f5b1932.slice/crio-0f66b373cf43476e91b9c8e0a1b779c039f966693eb94fe61d5b18f754f1be11 WatchSource:0}: Error finding container 0f66b373cf43476e91b9c8e0a1b779c039f966693eb94fe61d5b18f754f1be11: Status 404 returned error can't find the container with id 0f66b373cf43476e91b9c8e0a1b779c039f966693eb94fe61d5b18f754f1be11 Jul 14 11:07:56.793353 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:56.793280 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c7pwn" event={"ID":"08a7f632-942c-4b96-893e-ce794f5b1932","Type":"ContainerStarted","Data":"0f66b373cf43476e91b9c8e0a1b779c039f966693eb94fe61d5b18f754f1be11"} Jul 14 11:07:57.486159 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:57.486134 2581 scope.go:117] "RemoveContainer" containerID="6fdcbf189f9b8d6e7077662f37d9d3d746707309c578d2a26089047c05e2fc0e" Jul 14 11:07:57.800772 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:57.800658 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:07:57.800772 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:57.800749 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hrpl5" event={"ID":"bca091db-f43d-447d-beb1-24cb429d4aad","Type":"ContainerStarted","Data":"8c528786063c715f7f51ed116992672a6776cadbc9943cd00c6211fc1d5a0574"} Jul 14 11:07:57.801389 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:57.801303 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:57.818956 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:57.818902 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-hrpl5" podStartSLOduration=20.299215898 podStartE2EDuration="24.818885174s" totalImagesPullingTime="4.519669276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:07:57.818612436 +0000 UTC m=+68.801583272" watchObservedRunningTime="2026-07-14 11:07:57.818885174 +0000 UTC m=+68.801855989" Jul 14 11:07:58.156973 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:58.156935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:58.159387 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:58.159358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"image-registry-8657c45967-npz6g\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:07:58.212997 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:58.212964 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-hrpl5" Jul 14 11:07:58.285256 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:58.285221 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cn4jk\"" Jul 14 11:07:58.292913 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:07:58.292889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:08:00.211298 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.211228 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8657c45967-npz6g"] Jul 14 11:08:00.214628 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:00.213718 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e49fe4_ed80_4c3a_b73a_feb001215541.slice/crio-f75208e307966de6efed3c31dd842fc67ed8810aadc7579d0859a46a11b43b0f WatchSource:0}: Error finding container f75208e307966de6efed3c31dd842fc67ed8810aadc7579d0859a46a11b43b0f: Status 404 returned error can't find the container with id f75208e307966de6efed3c31dd842fc67ed8810aadc7579d0859a46a11b43b0f Jul 14 11:08:00.816003 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.815967 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mwx66" event={"ID":"60660683-0b39-4db6-a3ad-0261e7a7a741","Type":"ContainerStarted","Data":"6594b0b0014b8924f39e1d728abad86a2f0bfde70a1228ad15a4dbe951730e61"} Jul 14 11:08:00.817683 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.817652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c7pwn" event={"ID":"08a7f632-942c-4b96-893e-ce794f5b1932","Type":"ContainerStarted","Data":"83a705125d8873e2f2e04b45761bb7fab9cb0c89bd80ed69f212d2aa3a4cfd58"} Jul 14 11:08:00.817817 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.817702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c7pwn" event={"ID":"08a7f632-942c-4b96-893e-ce794f5b1932","Type":"ContainerStarted","Data":"5db7f5653c9cd9ed162b9e8f0b5c63c14ba60e1113ddc9b190de27237290610f"} Jul 14 11:08:00.817817 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.817733 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c7pwn" Jul 14 11:08:00.819017 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.818994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8657c45967-npz6g" event={"ID":"b6e49fe4-ed80-4c3a-b73a-feb001215541","Type":"ContainerStarted","Data":"d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62"} Jul 14 11:08:00.819158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.819025 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8657c45967-npz6g" event={"ID":"b6e49fe4-ed80-4c3a-b73a-feb001215541","Type":"ContainerStarted","Data":"f75208e307966de6efed3c31dd842fc67ed8810aadc7579d0859a46a11b43b0f"} Jul 14 11:08:00.819158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.819135 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:08:00.830148 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.830070 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mwx66" podStartSLOduration=67.297152189 podStartE2EDuration="1m11.83004088s" totalImagesPullingTime="4.532888691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:00.829963301 +0000 UTC m=+71.812934137" watchObservedRunningTime="2026-07-14 11:08:00.83004088 +0000 UTC m=+71.813011713" Jul 14 11:08:00.851067 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.851030 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8657c45967-npz6g" podStartSLOduration=18.851018095 podStartE2EDuration="18.851018095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:00.850929259 +0000 UTC m=+71.833900094" watchObservedRunningTime="2026-07-14 11:08:00.851018095 +0000 UTC m=+71.833988928" Jul 14 11:08:00.867687 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:00.867652 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c7pwn" podStartSLOduration=33.484312027 podStartE2EDuration="36.867640363s" totalImagesPullingTime="3.383328336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:00.867269879 +0000 UTC m=+71.850240715" watchObservedRunningTime="2026-07-14 11:08:00.867640363 +0000 UTC m=+71.850611196" Jul 14 11:08:04.845729 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.845703 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-kczsz"] Jul 14 11:08:04.848938 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.848911 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:04.851084 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.851059 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh"] Jul 14 11:08:04.853897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.853878 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:04.854636 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.854618 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 14 11:08:04.855150 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.855130 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 14 11:08:04.855425 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.855401 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9kgsk\"" Jul 14 11:08:04.859801 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.859783 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 11:08:04.860036 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.860020 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 11:08:04.860319 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.860294 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 11:08:04.860419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.860329 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-plxfn\"" Jul 14 11:08:04.860419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.860366 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 11:08:04.866578 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.866561 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-kczsz"] Jul 14 11:08:04.873203 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.873184 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh"] Jul 14 11:08:04.910563 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.910542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6bf9fd6-11a2-462b-abc0-cceca0ec3919-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh\" (UID: \"e6bf9fd6-11a2-462b-abc0-cceca0ec3919\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:04.910651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.910570 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/82f556b1-d504-4d9d-98be-d3cfd0485a15-nginx-conf\") pod \"networking-console-plugin-6cf856758c-kczsz\" (UID: \"82f556b1-d504-4d9d-98be-d3cfd0485a15\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:04.910651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.910588 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/82f556b1-d504-4d9d-98be-d3cfd0485a15-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kczsz\" (UID: \"82f556b1-d504-4d9d-98be-d3cfd0485a15\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:04.910718 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.910695 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4c94\" (UniqueName: \"kubernetes.io/projected/e6bf9fd6-11a2-462b-abc0-cceca0ec3919-kube-api-access-t4c94\") pod \"managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh\" (UID: \"e6bf9fd6-11a2-462b-abc0-cceca0ec3919\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:04.945274 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.945252 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8657c45967-npz6g"] Jul 14 11:08:04.991441 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.991412 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6948cf4b79-hhcdx"] Jul 14 11:08:04.993497 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:04.993479 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.009477 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.009456 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6948cf4b79-hhcdx"] Jul 14 11:08:05.011650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.011627 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4c94\" (UniqueName: \"kubernetes.io/projected/e6bf9fd6-11a2-462b-abc0-cceca0ec3919-kube-api-access-t4c94\") pod \"managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh\" (UID: \"e6bf9fd6-11a2-462b-abc0-cceca0ec3919\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:05.011753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.011669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6bf9fd6-11a2-462b-abc0-cceca0ec3919-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh\" (UID: \"e6bf9fd6-11a2-462b-abc0-cceca0ec3919\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:05.011753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.011693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/82f556b1-d504-4d9d-98be-d3cfd0485a15-nginx-conf\") pod \"networking-console-plugin-6cf856758c-kczsz\" (UID: \"82f556b1-d504-4d9d-98be-d3cfd0485a15\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:05.011862 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.011803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/82f556b1-d504-4d9d-98be-d3cfd0485a15-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kczsz\" (UID: \"82f556b1-d504-4d9d-98be-d3cfd0485a15\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:05.012277 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.012254 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/82f556b1-d504-4d9d-98be-d3cfd0485a15-nginx-conf\") pod \"networking-console-plugin-6cf856758c-kczsz\" (UID: \"82f556b1-d504-4d9d-98be-d3cfd0485a15\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:05.014015 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.013987 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6bf9fd6-11a2-462b-abc0-cceca0ec3919-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh\" (UID: \"e6bf9fd6-11a2-462b-abc0-cceca0ec3919\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:05.014132 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.014061 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/82f556b1-d504-4d9d-98be-d3cfd0485a15-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kczsz\" (UID: \"82f556b1-d504-4d9d-98be-d3cfd0485a15\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:05.025399 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.025379 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4c94\" (UniqueName: \"kubernetes.io/projected/e6bf9fd6-11a2-462b-abc0-cceca0ec3919-kube-api-access-t4c94\") pod \"managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh\" (UID: \"e6bf9fd6-11a2-462b-abc0-cceca0ec3919\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:05.112337 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112286 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bd99376-af9d-4ec8-906e-82f00bbad8cc-ca-trust-extracted\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112337 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bd99376-af9d-4ec8-906e-82f00bbad8cc-trusted-ca\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112464 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-registry-tls\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112464 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bd99376-af9d-4ec8-906e-82f00bbad8cc-registry-certificates\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112464 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112459 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bd99376-af9d-4ec8-906e-82f00bbad8cc-image-registry-private-configuration\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112562 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112487 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-bound-sa-token\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112562 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvccp\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-kube-api-access-lvccp\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.112562 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.112528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bd99376-af9d-4ec8-906e-82f00bbad8cc-installation-pull-secrets\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.160906 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.160884 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" Jul 14 11:08:05.172304 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.172281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" Jul 14 11:08:05.213361 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213335 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bd99376-af9d-4ec8-906e-82f00bbad8cc-registry-certificates\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213381 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bd99376-af9d-4ec8-906e-82f00bbad8cc-image-registry-private-configuration\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213413 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-bound-sa-token\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213432 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvccp\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-kube-api-access-lvccp\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bd99376-af9d-4ec8-906e-82f00bbad8cc-installation-pull-secrets\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bd99376-af9d-4ec8-906e-82f00bbad8cc-ca-trust-extracted\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213521 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bd99376-af9d-4ec8-906e-82f00bbad8cc-trusted-ca\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.213850 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.213547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-registry-tls\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.215809 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.214839 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bd99376-af9d-4ec8-906e-82f00bbad8cc-registry-certificates\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.215809 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.214885 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bd99376-af9d-4ec8-906e-82f00bbad8cc-ca-trust-extracted\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.215809 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.215719 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bd99376-af9d-4ec8-906e-82f00bbad8cc-trusted-ca\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.216749 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.216710 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bd99376-af9d-4ec8-906e-82f00bbad8cc-image-registry-private-configuration\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.217381 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.217363 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bd99376-af9d-4ec8-906e-82f00bbad8cc-installation-pull-secrets\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.217557 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.217534 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-registry-tls\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.227785 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.227630 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvccp\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-kube-api-access-lvccp\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.231044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.231001 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bd99376-af9d-4ec8-906e-82f00bbad8cc-bound-sa-token\") pod \"image-registry-6948cf4b79-hhcdx\" (UID: \"8bd99376-af9d-4ec8-906e-82f00bbad8cc\") " pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.293904 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.293876 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-kczsz"] Jul 14 11:08:05.297214 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:05.297191 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f556b1_d504_4d9d_98be_d3cfd0485a15.slice/crio-53e07db6a843cff8f88ee66d64e16a2beec8b0eaf87c522bfffda359138fbf05 WatchSource:0}: Error finding container 53e07db6a843cff8f88ee66d64e16a2beec8b0eaf87c522bfffda359138fbf05: Status 404 returned error can't find the container with id 53e07db6a843cff8f88ee66d64e16a2beec8b0eaf87c522bfffda359138fbf05 Jul 14 11:08:05.302750 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.302729 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.320760 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.320727 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh"] Jul 14 11:08:05.324364 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:05.324334 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6bf9fd6_11a2_462b_abc0_cceca0ec3919.slice/crio-7b7d217c5d9af674f550cad0bf3580559410dd221288704121b9810fe2e4b4a8 WatchSource:0}: Error finding container 7b7d217c5d9af674f550cad0bf3580559410dd221288704121b9810fe2e4b4a8: Status 404 returned error can't find the container with id 7b7d217c5d9af674f550cad0bf3580559410dd221288704121b9810fe2e4b4a8 Jul 14 11:08:05.414928 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.414897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:08:05.417048 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.417032 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6986c71b-b111-413a-95e7-30a00b1d2068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cf46g\" (UID: \"6986c71b-b111-413a-95e7-30a00b1d2068\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:08:05.432962 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.432941 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6948cf4b79-hhcdx"] Jul 14 11:08:05.435491 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:05.435470 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd99376_af9d_4ec8_906e_82f00bbad8cc.slice/crio-711031e4accdcab6599e1b69725e42ace5cafcab70c66e2b0997380dd4a9b194 WatchSource:0}: Error finding container 711031e4accdcab6599e1b69725e42ace5cafcab70c66e2b0997380dd4a9b194: Status 404 returned error can't find the container with id 711031e4accdcab6599e1b69725e42ace5cafcab70c66e2b0997380dd4a9b194 Jul 14 11:08:05.610354 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.610314 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9tdmz\"" Jul 14 11:08:05.619180 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.619161 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" Jul 14 11:08:05.743297 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.743266 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g"] Jul 14 11:08:05.745744 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:05.745720 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6986c71b_b111_413a_95e7_30a00b1d2068.slice/crio-ffd1fd7ca7461b33b2416ac935312dbbca929dfe0c7810692c48ec3fd5f074f6 WatchSource:0}: Error finding container ffd1fd7ca7461b33b2416ac935312dbbca929dfe0c7810692c48ec3fd5f074f6: Status 404 returned error can't find the container with id ffd1fd7ca7461b33b2416ac935312dbbca929dfe0c7810692c48ec3fd5f074f6 Jul 14 11:08:05.836929 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.836898 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" event={"ID":"82f556b1-d504-4d9d-98be-d3cfd0485a15","Type":"ContainerStarted","Data":"53e07db6a843cff8f88ee66d64e16a2beec8b0eaf87c522bfffda359138fbf05"} Jul 14 11:08:05.838024 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.837996 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" event={"ID":"6986c71b-b111-413a-95e7-30a00b1d2068","Type":"ContainerStarted","Data":"ffd1fd7ca7461b33b2416ac935312dbbca929dfe0c7810692c48ec3fd5f074f6"} Jul 14 11:08:05.839043 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.839018 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" event={"ID":"e6bf9fd6-11a2-462b-abc0-cceca0ec3919","Type":"ContainerStarted","Data":"7b7d217c5d9af674f550cad0bf3580559410dd221288704121b9810fe2e4b4a8"} Jul 14 11:08:05.840233 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.840210 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" event={"ID":"8bd99376-af9d-4ec8-906e-82f00bbad8cc","Type":"ContainerStarted","Data":"da95a884e1b4d96590794e43a4cb02e26e1ecc9ea7f978ebf3f4616b1fad4937"} Jul 14 11:08:05.840306 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.840252 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" event={"ID":"8bd99376-af9d-4ec8-906e-82f00bbad8cc","Type":"ContainerStarted","Data":"711031e4accdcab6599e1b69725e42ace5cafcab70c66e2b0997380dd4a9b194"} Jul 14 11:08:05.840351 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.840336 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:05.858982 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:05.858938 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" podStartSLOduration=1.858920141 podStartE2EDuration="1.858920141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:05.857869031 +0000 UTC m=+76.840839868" watchObservedRunningTime="2026-07-14 11:08:05.858920141 +0000 UTC m=+76.841890970" Jul 14 11:08:06.849323 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:06.849249 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" event={"ID":"82f556b1-d504-4d9d-98be-d3cfd0485a15","Type":"ContainerStarted","Data":"a7423edb5ae693376986ef36367321108660dbe2af565c84097546effe399b53"} Jul 14 11:08:06.866588 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:06.866000 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-kczsz" podStartSLOduration=1.553524133 podStartE2EDuration="2.86597679s" totalImagesPullingTime="1.312452657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:06.864507278 +0000 UTC m=+77.847478111" watchObservedRunningTime="2026-07-14 11:08:06.86597679 +0000 UTC m=+77.848947624" Jul 14 11:08:08.857958 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:08.857906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" event={"ID":"6986c71b-b111-413a-95e7-30a00b1d2068","Type":"ContainerStarted","Data":"492853dc97e7fcab72a2d72a408f1c9f8c0772dcea4c8e4a7aa549d404b846d7"} Jul 14 11:08:08.859710 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:08.859684 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" event={"ID":"e6bf9fd6-11a2-462b-abc0-cceca0ec3919","Type":"ContainerStarted","Data":"c514650011259f21808c9a30f30919b36bd797fcb78fb6ff3ed2ecdb6c93d2da"} Jul 14 11:08:08.876613 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:08.876573 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cf46g" podStartSLOduration=33.259447217 podStartE2EDuration="35.876560627s" totalImagesPullingTime="2.61711341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:08.876095999 +0000 UTC m=+79.859066832" watchObservedRunningTime="2026-07-14 11:08:08.876560627 +0000 UTC m=+79.859531461" Jul 14 11:08:08.896287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:08.896244 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c88cd5c9d-kmdkh" podStartSLOduration=1.8568134710000002 podStartE2EDuration="4.896227618s" totalImagesPullingTime="3.039414147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:08.895430075 +0000 UTC m=+79.878400909" watchObservedRunningTime="2026-07-14 11:08:08.896227618 +0000 UTC m=+79.879198453" Jul 14 11:08:09.353728 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:09.353690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:08:09.355777 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:09.355759 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/091c0e73-d9d2-4892-8912-3b249e1dfcbb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7sf5x\" (UID: \"091c0e73-d9d2-4892-8912-3b249e1dfcbb\") " pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:08:09.572491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:09.572465 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-24zbp\"" Jul 14 11:08:09.581065 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:09.581049 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7sf5x" Jul 14 11:08:09.703607 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:09.703582 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7sf5x"] Jul 14 11:08:09.706851 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:09.706825 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091c0e73_d9d2_4892_8912_3b249e1dfcbb.slice/crio-c29f3f93ca3ee67a23d08b7c96e0df8faaa15a60f02f1bf26003c4880ef39dd7 WatchSource:0}: Error finding container c29f3f93ca3ee67a23d08b7c96e0df8faaa15a60f02f1bf26003c4880ef39dd7: Status 404 returned error can't find the container with id c29f3f93ca3ee67a23d08b7c96e0df8faaa15a60f02f1bf26003c4880ef39dd7 Jul 14 11:08:09.863765 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:09.863737 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sf5x" event={"ID":"091c0e73-d9d2-4892-8912-3b249e1dfcbb","Type":"ContainerStarted","Data":"c29f3f93ca3ee67a23d08b7c96e0df8faaa15a60f02f1bf26003c4880ef39dd7"} Jul 14 11:08:10.824732 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:10.824706 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c7pwn" Jul 14 11:08:12.384499 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.384467 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d8cf5f657-2k4ts"] Jul 14 11:08:12.584489 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.584459 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8cf5f657-2k4ts"] Jul 14 11:08:12.584844 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.584564 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.586767 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.586742 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 11:08:12.586897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.586845 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 11:08:12.586949 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.586925 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sk9qm\"" Jul 14 11:08:12.587303 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.587281 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 11:08:12.587395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.587285 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 11:08:12.587395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.587341 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 11:08:12.587395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.587362 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 11:08:12.587562 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.587409 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 11:08:12.680682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.680608 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-oauth-config\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.680682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.680657 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-serving-cert\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.680682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.680679 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-service-ca\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.680897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.680837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-console-config\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.680897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.680863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-oauth-serving-cert\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.680897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.680878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhcgh\" (UniqueName: \"kubernetes.io/projected/b72681f5-580d-401b-a6e4-536ece78c8ae-kube-api-access-bhcgh\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782012 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.781978 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-serving-cert\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782012 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-service-ca\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782070 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-console-config\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-oauth-serving-cert\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782102 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcgh\" (UniqueName: \"kubernetes.io/projected/b72681f5-580d-401b-a6e4-536ece78c8ae-kube-api-access-bhcgh\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782262 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-oauth-config\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782783 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782751 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-console-config\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782887 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782859 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-service-ca\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.782926 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.782859 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-oauth-serving-cert\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.784355 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.784326 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-oauth-config\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.784505 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.784486 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-serving-cert\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.791271 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.791228 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhcgh\" (UniqueName: \"kubernetes.io/projected/b72681f5-580d-401b-a6e4-536ece78c8ae-kube-api-access-bhcgh\") pod \"console-6d8cf5f657-2k4ts\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:12.894453 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:12.894422 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:13.781763 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:13.781739 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8cf5f657-2k4ts"] Jul 14 11:08:13.784351 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:13.784307 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72681f5_580d_401b_a6e4_536ece78c8ae.slice/crio-29d9e100f1bcd7ce81788a1b9e9acb7a345e3b543f090b1ce5aedc1af797b7a1 WatchSource:0}: Error finding container 29d9e100f1bcd7ce81788a1b9e9acb7a345e3b543f090b1ce5aedc1af797b7a1: Status 404 returned error can't find the container with id 29d9e100f1bcd7ce81788a1b9e9acb7a345e3b543f090b1ce5aedc1af797b7a1 Jul 14 11:08:13.882841 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:13.882805 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sf5x" event={"ID":"091c0e73-d9d2-4892-8912-3b249e1dfcbb","Type":"ContainerStarted","Data":"ad51a251119b5c40b9d798c648d9176b641156bf0c0306bf1d3fc6fe8bc57031"} Jul 14 11:08:13.883987 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:13.883962 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8cf5f657-2k4ts" event={"ID":"b72681f5-580d-401b-a6e4-536ece78c8ae","Type":"ContainerStarted","Data":"29d9e100f1bcd7ce81788a1b9e9acb7a345e3b543f090b1ce5aedc1af797b7a1"} Jul 14 11:08:14.892504 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:14.892469 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sf5x" event={"ID":"091c0e73-d9d2-4892-8912-3b249e1dfcbb","Type":"ContainerStarted","Data":"7f92dea9b1e70a33eb5494a800fc1117dc2d72628e9c5cb33d966777173b9017"} Jul 14 11:08:17.906378 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:17.906331 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7sf5x" event={"ID":"091c0e73-d9d2-4892-8912-3b249e1dfcbb","Type":"ContainerStarted","Data":"846e98c5306475078b5859418c84cdcb635d565050001f86b6911d1616965277"} Jul 14 11:08:17.908016 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:17.907994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8cf5f657-2k4ts" event={"ID":"b72681f5-580d-401b-a6e4-536ece78c8ae","Type":"ContainerStarted","Data":"d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8"} Jul 14 11:08:17.924544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:17.924497 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7sf5x" podStartSLOduration=32.975583905 podStartE2EDuration="40.924483611s" totalImagesPullingTime="7.948899706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:07:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:17.923168575 +0000 UTC m=+88.906139419" watchObservedRunningTime="2026-07-14 11:08:17.924483611 +0000 UTC m=+88.907454445" Jul 14 11:08:17.943062 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:17.942834 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d8cf5f657-2k4ts" podStartSLOduration=1.96443065 podStartE2EDuration="5.942816891s" totalImagesPullingTime="3.978386241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:17.941859328 +0000 UTC m=+88.924830162" watchObservedRunningTime="2026-07-14 11:08:17.942816891 +0000 UTC m=+88.925787725" Jul 14 11:08:18.301609 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.301577 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-jg2xc"] Jul 14 11:08:18.303859 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.303840 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.307349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.307329 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 14 11:08:18.307349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.307340 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 11:08:18.307528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.307349 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 14 11:08:18.307528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.307372 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hwnkm\"" Jul 14 11:08:18.307528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.307459 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 14 11:08:18.313435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.313415 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-msl2k"] Jul 14 11:08:18.319496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.319477 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.321974 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.321952 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 11:08:18.322167 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.321982 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 11:08:18.322167 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.321984 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7hgj\"" Jul 14 11:08:18.322329 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.322294 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 11:08:18.322530 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.322433 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-jg2xc"] Jul 14 11:08:18.433668 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433634 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-metrics-client-ca\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.433843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrwn\" (UniqueName: \"kubernetes.io/projected/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-kube-api-access-ncrwn\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.433843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433713 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.433843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433768 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c539d70-c00b-49c0-b121-3862b52eb80b-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.433843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-wtmp\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433904 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-textfile\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-sys\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433939 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433979 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.433996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-root\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434303 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.434012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-tls\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.434303 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.434035 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8bsz\" (UniqueName: \"kubernetes.io/projected/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-api-access-n8bsz\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.434303 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.434091 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c539d70-c00b-49c0-b121-3862b52eb80b-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.535308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535270 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-textfile\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535312 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-sys\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535396 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-sys\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-root\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-tls\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8bsz\" (UniqueName: \"kubernetes.io/projected/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-api-access-n8bsz\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.535897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535572 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c539d70-c00b-49c0-b121-3862b52eb80b-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.535897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535605 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-root\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-metrics-client-ca\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.535897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535630 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-textfile\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536092 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535938 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrwn\" (UniqueName: \"kubernetes.io/projected/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-kube-api-access-ncrwn\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536092 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.535999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.536092 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c539d70-c00b-49c0-b121-3862b52eb80b-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.536092 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-wtmp\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536314 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536314 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-metrics-client-ca\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536314 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536201 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536462 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536413 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-wtmp\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.536462 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536427 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c539d70-c00b-49c0-b121-3862b52eb80b-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.536595 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536570 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c539d70-c00b-49c0-b121-3862b52eb80b-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.536899 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.536875 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.538143 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.538093 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.538261 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.538215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.538326 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.538276 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-tls\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.538420 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.538404 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.543498 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.543476 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8bsz\" (UniqueName: \"kubernetes.io/projected/2c539d70-c00b-49c0-b121-3862b52eb80b-kube-api-access-n8bsz\") pod \"kube-state-metrics-785778ff74-jg2xc\" (UID: \"2c539d70-c00b-49c0-b121-3862b52eb80b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.543613 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.543510 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrwn\" (UniqueName: \"kubernetes.io/projected/6d3936f7-54ac-4ab3-bf2e-dba215e060a6-kube-api-access-ncrwn\") pod \"node-exporter-msl2k\" (UID: \"6d3936f7-54ac-4ab3-bf2e-dba215e060a6\") " pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.612906 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.612831 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" Jul 14 11:08:18.628633 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.628610 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-msl2k" Jul 14 11:08:18.637670 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:18.637636 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3936f7_54ac_4ab3_bf2e_dba215e060a6.slice/crio-027259bad0abafc7266cb2c9088f5ab275f9bd85c3d3b3249878fe40d8f6fce7 WatchSource:0}: Error finding container 027259bad0abafc7266cb2c9088f5ab275f9bd85c3d3b3249878fe40d8f6fce7: Status 404 returned error can't find the container with id 027259bad0abafc7266cb2c9088f5ab275f9bd85c3d3b3249878fe40d8f6fce7 Jul 14 11:08:18.741264 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.741231 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-jg2xc"] Jul 14 11:08:18.744683 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:18.744660 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c539d70_c00b_49c0_b121_3862b52eb80b.slice/crio-24c25cd685d214e7a1155e932946b61f7e1a15730f1915de847ec545059b7334 WatchSource:0}: Error finding container 24c25cd685d214e7a1155e932946b61f7e1a15730f1915de847ec545059b7334: Status 404 returned error can't find the container with id 24c25cd685d214e7a1155e932946b61f7e1a15730f1915de847ec545059b7334 Jul 14 11:08:18.912032 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.912000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msl2k" event={"ID":"6d3936f7-54ac-4ab3-bf2e-dba215e060a6","Type":"ContainerStarted","Data":"027259bad0abafc7266cb2c9088f5ab275f9bd85c3d3b3249878fe40d8f6fce7"} Jul 14 11:08:18.913192 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:18.913151 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" event={"ID":"2c539d70-c00b-49c0-b121-3862b52eb80b","Type":"ContainerStarted","Data":"24c25cd685d214e7a1155e932946b61f7e1a15730f1915de847ec545059b7334"} Jul 14 11:08:19.918902 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:19.918833 2581 generic.go:356] "Generic (PLEG): container finished" podID="6d3936f7-54ac-4ab3-bf2e-dba215e060a6" containerID="cf8a2d5ab539354b2925d256a214f98f5aeb4597585a7cda3d0e260832ef99a7" exitCode=0 Jul 14 11:08:19.919256 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:19.918905 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msl2k" event={"ID":"6d3936f7-54ac-4ab3-bf2e-dba215e060a6","Type":"ContainerDied","Data":"cf8a2d5ab539354b2925d256a214f98f5aeb4597585a7cda3d0e260832ef99a7"} Jul 14 11:08:20.361088 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.361057 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5445b8485d-gf7cz"] Jul 14 11:08:20.366384 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.366358 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.368961 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.368566 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 11:08:20.368961 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.368911 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 11:08:20.369502 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.369475 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 11:08:20.369502 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.369495 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-210476jbu3j98\"" Jul 14 11:08:20.369880 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.369862 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 11:08:20.370139 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.370048 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lhkj7\"" Jul 14 11:08:20.370248 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.370163 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 11:08:20.375002 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.374984 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5445b8485d-gf7cz"] Jul 14 11:08:20.452853 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.452817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.452982 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.452930 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghqwx\" (UniqueName: \"kubernetes.io/projected/54ef4e74-4697-462e-9348-0a64f3369b7d-kube-api-access-ghqwx\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.453024 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.452986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-tls\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.453024 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.453006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.453103 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.453029 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.453103 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.453090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-grpc-tls\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.453232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.453149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54ef4e74-4697-462e-9348-0a64f3369b7d-metrics-client-ca\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.453232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.453203 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghqwx\" (UniqueName: \"kubernetes.io/projected/54ef4e74-4697-462e-9348-0a64f3369b7d-kube-api-access-ghqwx\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554656 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-tls\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554886 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554677 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554886 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554696 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554886 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-grpc-tls\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554886 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554744 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54ef4e74-4697-462e-9348-0a64f3369b7d-metrics-client-ca\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.554886 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554763 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.555202 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.554952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.555737 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.555710 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54ef4e74-4697-462e-9348-0a64f3369b7d-metrics-client-ca\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.557644 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.557614 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.557762 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.557719 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-grpc-tls\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.557826 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.557760 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.557826 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.557796 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.557826 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.557811 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.557930 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.557892 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/54ef4e74-4697-462e-9348-0a64f3369b7d-secret-thanos-querier-tls\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.566163 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.566144 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghqwx\" (UniqueName: \"kubernetes.io/projected/54ef4e74-4697-462e-9348-0a64f3369b7d-kube-api-access-ghqwx\") pod \"thanos-querier-5445b8485d-gf7cz\" (UID: \"54ef4e74-4697-462e-9348-0a64f3369b7d\") " pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.689640 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.689614 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:20.830016 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.829984 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5445b8485d-gf7cz"] Jul 14 11:08:20.833841 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:20.833810 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ef4e74_4697_462e_9348_0a64f3369b7d.slice/crio-aca31eacf8eacf2fbd26fa079fcfddc188d3c0d916c089e278a40fe287d3b28d WatchSource:0}: Error finding container aca31eacf8eacf2fbd26fa079fcfddc188d3c0d916c089e278a40fe287d3b28d: Status 404 returned error can't find the container with id aca31eacf8eacf2fbd26fa079fcfddc188d3c0d916c089e278a40fe287d3b28d Jul 14 11:08:20.925339 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.925300 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" event={"ID":"2c539d70-c00b-49c0-b121-3862b52eb80b","Type":"ContainerStarted","Data":"35ac78e4495aa33be775c0a0b2b857b6a9a656139b52902e3345b419faaff3e1"} Jul 14 11:08:20.925753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.925352 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" event={"ID":"2c539d70-c00b-49c0-b121-3862b52eb80b","Type":"ContainerStarted","Data":"26a9aa1689ff49d4f29b97d402ee876d684263bee7092dae0c66a1a92e9a41e1"} Jul 14 11:08:20.925753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.925371 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" event={"ID":"2c539d70-c00b-49c0-b121-3862b52eb80b","Type":"ContainerStarted","Data":"18ca925bb6c38b86d97985724b0dfc994e59e90a38db5aa6ab9c847e91855909"} Jul 14 11:08:20.926492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.926466 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"aca31eacf8eacf2fbd26fa079fcfddc188d3c0d916c089e278a40fe287d3b28d"} Jul 14 11:08:20.928492 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.928471 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msl2k" event={"ID":"6d3936f7-54ac-4ab3-bf2e-dba215e060a6","Type":"ContainerStarted","Data":"3777a5378496b208f41c60c627515a98209cdcd7b7cc99459dc0a8d216d44103"} Jul 14 11:08:20.928578 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.928508 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msl2k" event={"ID":"6d3936f7-54ac-4ab3-bf2e-dba215e060a6","Type":"ContainerStarted","Data":"2f6929dbcc32127241d14c7ab17455318d0eff82ecca9421748eb5d9e8731e91"} Jul 14 11:08:20.947163 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.947107 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-jg2xc" podStartSLOduration=1.427270681 podStartE2EDuration="2.947092875s" totalImagesPullingTime="1.519822194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:20.944736747 +0000 UTC m=+91.927707580" watchObservedRunningTime="2026-07-14 11:08:20.947092875 +0000 UTC m=+91.930063748" Jul 14 11:08:20.965186 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:20.965101 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-msl2k" podStartSLOduration=1.872890398 podStartE2EDuration="2.965085551s" totalImagesPullingTime="1.02303544s" totalInitContainerRuntime="69.159713ms" isStatefulPod=true podCreationTimestamp="2026-07-14 11:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:20.96365532 +0000 UTC m=+91.946626166" watchObservedRunningTime="2026-07-14 11:08:20.965085551 +0000 UTC m=+91.948056390" Jul 14 11:08:22.679654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.679618 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-54885f4554-wt9dp"] Jul 14 11:08:22.683347 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.683327 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.685872 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.685843 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2sf2d\"" Jul 14 11:08:22.686291 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.686252 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 11:08:22.686650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.686463 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 11:08:22.686650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.686528 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3meo0n6crqqv7\"" Jul 14 11:08:22.686650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.686552 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 11:08:22.686650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.686529 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 11:08:22.695579 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.695556 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54885f4554-wt9dp"] Jul 14 11:08:22.776884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.776851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-audit-log\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.777066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.776895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-client-ca-bundle\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.777066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.776938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-secret-metrics-server-client-certs\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.777066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.776972 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-secret-metrics-server-tls\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.777066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.777007 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-metrics-server-audit-profiles\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.777293 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.777096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22w9h\" (UniqueName: \"kubernetes.io/projected/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-kube-api-access-22w9h\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.777293 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.777186 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22w9h\" (UniqueName: \"kubernetes.io/projected/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-kube-api-access-22w9h\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-audit-log\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-client-ca-bundle\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878356 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-secret-metrics-server-client-certs\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878404 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-secret-metrics-server-tls\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878680 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-metrics-server-audit-profiles\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.878788 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.878767 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-audit-log\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.879171 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.879148 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.879460 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.879441 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-metrics-server-audit-profiles\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.881371 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.881350 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-secret-metrics-server-tls\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.881457 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.881440 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-secret-metrics-server-client-certs\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.881559 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.881537 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-client-ca-bundle\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.887286 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.887261 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22w9h\" (UniqueName: \"kubernetes.io/projected/fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181-kube-api-access-22w9h\") pod \"metrics-server-54885f4554-wt9dp\" (UID: \"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181\") " pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:22.894963 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.894929 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:22.895059 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.894988 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:22.900242 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.900225 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:22.943168 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.943087 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:08:22.993656 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:22.993622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:23.107870 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.107809 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f94f6c8cd-m6vzs"] Jul 14 11:08:23.111199 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.111159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.118452 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.118297 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 11:08:23.123721 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.123700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f94f6c8cd-m6vzs"] Jul 14 11:08:23.188708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.188683 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54885f4554-wt9dp"] Jul 14 11:08:23.192319 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:23.192296 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8e69bc_5c50_4ccc_8dc0_91d55fbd9181.slice/crio-0aa34e47d586b8a4c3d2a8ea81680098e20c2867e27a69b0a02c1ba4da8a9eba WatchSource:0}: Error finding container 0aa34e47d586b8a4c3d2a8ea81680098e20c2867e27a69b0a02c1ba4da8a9eba: Status 404 returned error can't find the container with id 0aa34e47d586b8a4c3d2a8ea81680098e20c2867e27a69b0a02c1ba4da8a9eba Jul 14 11:08:23.283981 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.283954 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-serving-cert\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.284090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.283995 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-oauth-serving-cert\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.284090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.284027 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-oauth-config\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.284090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.284045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-console-config\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.284090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.284073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-trusted-ca-bundle\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.284090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.284090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-service-ca\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.284281 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.284144 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswwx\" (UniqueName: \"kubernetes.io/projected/12867806-1ad5-4eee-ac76-66ab1079d987-kube-api-access-cswwx\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385308 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-serving-cert\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385328 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-oauth-serving-cert\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385365 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-oauth-config\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385392 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-console-config\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-trusted-ca-bundle\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385691 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-service-ca\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.385691 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.385516 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cswwx\" (UniqueName: \"kubernetes.io/projected/12867806-1ad5-4eee-ac76-66ab1079d987-kube-api-access-cswwx\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.386240 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.386216 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-console-config\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.386240 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.386236 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-service-ca\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.386460 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.386434 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-trusted-ca-bundle\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.386920 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.386882 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-oauth-serving-cert\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.387974 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.387957 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-oauth-config\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.388131 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.388095 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-serving-cert\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.392429 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.392410 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswwx\" (UniqueName: \"kubernetes.io/projected/12867806-1ad5-4eee-ac76-66ab1079d987-kube-api-access-cswwx\") pod \"console-f94f6c8cd-m6vzs\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.426155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.423286 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:23.563135 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.563088 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f94f6c8cd-m6vzs"] Jul 14 11:08:23.566484 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:23.566456 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12867806_1ad5_4eee_ac76_66ab1079d987.slice/crio-98a43ad4ec9d09e9d3f83c69d05804232c6b8eab6b0fdc0b39c4f1b43a2c2fe8 WatchSource:0}: Error finding container 98a43ad4ec9d09e9d3f83c69d05804232c6b8eab6b0fdc0b39c4f1b43a2c2fe8: Status 404 returned error can't find the container with id 98a43ad4ec9d09e9d3f83c69d05804232c6b8eab6b0fdc0b39c4f1b43a2c2fe8 Jul 14 11:08:23.946500 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.946450 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94f6c8cd-m6vzs" event={"ID":"12867806-1ad5-4eee-ac76-66ab1079d987","Type":"ContainerStarted","Data":"6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a"} Jul 14 11:08:23.946937 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.946552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94f6c8cd-m6vzs" event={"ID":"12867806-1ad5-4eee-ac76-66ab1079d987","Type":"ContainerStarted","Data":"98a43ad4ec9d09e9d3f83c69d05804232c6b8eab6b0fdc0b39c4f1b43a2c2fe8"} Jul 14 11:08:23.949624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.949593 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"1008acf089f4a6e4218f77aac364f6c13048d70a2f90d6be9b6ac117342fc57c"} Jul 14 11:08:23.949921 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.949898 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"cab316eb4554e25117157a50a28eabc9992d628c658bfedc143471aa197a07b7"} Jul 14 11:08:23.950099 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.950083 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"7e7323abf9e0891fb5efbea90bf7303308fefd78b9e7c575dfdfe09225928e36"} Jul 14 11:08:23.951210 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.951178 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" event={"ID":"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181","Type":"ContainerStarted","Data":"0aa34e47d586b8a4c3d2a8ea81680098e20c2867e27a69b0a02c1ba4da8a9eba"} Jul 14 11:08:23.965158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:23.965086 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f94f6c8cd-m6vzs" podStartSLOduration=0.96507215 podStartE2EDuration="965.07215ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:23.963218519 +0000 UTC m=+94.946189365" watchObservedRunningTime="2026-07-14 11:08:23.96507215 +0000 UTC m=+94.948043188" Jul 14 11:08:24.951367 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.951343 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:08:24.958250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.958218 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"f2c9fc8e9811631a5f8ad69f93527c813c40d69f59f95cd6b092ec8fe759b402"} Jul 14 11:08:24.958435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.958295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"07817ba9eb97d00b99a78251f1927eb30dcd1366ab24c87b38d43c2ddd65f08e"} Jul 14 11:08:24.958435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.958339 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" event={"ID":"54ef4e74-4697-462e-9348-0a64f3369b7d","Type":"ContainerStarted","Data":"73f22ecca1b26859a15c037116d266c1f13e81470420a749b304c96dc79eeabc"} Jul 14 11:08:24.958435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.958377 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:24.959634 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.959611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" event={"ID":"fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181","Type":"ContainerStarted","Data":"63e37f6e09d4c09782b0b419ad812e4ea1594ebcf2d5c0b666828b1e4089ee2f"} Jul 14 11:08:24.988303 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:24.988257 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" podStartSLOduration=1.330445216 podStartE2EDuration="2.988242221s" totalImagesPullingTime="1.657797005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:24.986778928 +0000 UTC m=+95.969749766" watchObservedRunningTime="2026-07-14 11:08:24.988242221 +0000 UTC m=+95.971213055" Jul 14 11:08:25.016463 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:25.016418 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" podStartSLOduration=1.8461373509999999 podStartE2EDuration="5.016400245s" totalImagesPullingTime="3.170262894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:25.01416528 +0000 UTC m=+95.997136109" watchObservedRunningTime="2026-07-14 11:08:25.016400245 +0000 UTC m=+95.999371080" Jul 14 11:08:26.796095 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:26.796069 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zc6px" Jul 14 11:08:26.854006 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:26.853978 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6948cf4b79-hhcdx" Jul 14 11:08:29.964913 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:29.964863 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8657c45967-npz6g" podUID="b6e49fe4-ed80-4c3a-b73a-feb001215541" containerName="registry" containerID="cri-o://d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62" gracePeriod=30 Jul 14 11:08:30.236960 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.236880 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:08:30.246696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246668 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-installation-pull-secrets\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.246839 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246709 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-bound-sa-token\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.246839 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246732 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-certificates\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.246839 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246758 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdmdd\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-kube-api-access-kdmdd\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.246839 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246790 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e49fe4-ed80-4c3a-b73a-feb001215541-ca-trust-extracted\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.246839 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.247062 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246870 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-image-registry-private-configuration\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.247062 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.246917 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-trusted-ca\") pod \"b6e49fe4-ed80-4c3a-b73a-feb001215541\" (UID: \"b6e49fe4-ed80-4c3a-b73a-feb001215541\") " Jul 14 11:08:30.247265 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.247236 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:08:30.247534 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.247507 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:08:30.249415 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.249386 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-kube-api-access-kdmdd" (OuterVolumeSpecName: "kube-api-access-kdmdd") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "kube-api-access-kdmdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:08:30.249624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.249594 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:08:30.249624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.249607 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:08:30.249762 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.249679 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:08:30.249762 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.249687 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:08:30.255942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.255878 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e49fe4-ed80-4c3a-b73a-feb001215541-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b6e49fe4-ed80-4c3a-b73a-feb001215541" (UID: "b6e49fe4-ed80-4c3a-b73a-feb001215541"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:08:30.348390 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348353 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-trusted-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348390 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348382 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-installation-pull-secrets\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348390 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348393 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-bound-sa-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348618 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348403 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-certificates\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348618 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348412 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdmdd\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-kube-api-access-kdmdd\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348618 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348420 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e49fe4-ed80-4c3a-b73a-feb001215541-ca-trust-extracted\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348618 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348428 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e49fe4-ed80-4c3a-b73a-feb001215541-registry-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.348618 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.348438 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e49fe4-ed80-4c3a-b73a-feb001215541-image-registry-private-configuration\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:30.969166 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.969132 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5445b8485d-gf7cz" Jul 14 11:08:30.983743 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.983712 2581 generic.go:356] "Generic (PLEG): container finished" podID="b6e49fe4-ed80-4c3a-b73a-feb001215541" containerID="d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62" exitCode=0 Jul 14 11:08:30.983876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.983751 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8657c45967-npz6g" event={"ID":"b6e49fe4-ed80-4c3a-b73a-feb001215541","Type":"ContainerDied","Data":"d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62"} Jul 14 11:08:30.983876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.983761 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8657c45967-npz6g" Jul 14 11:08:30.983876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.983786 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8657c45967-npz6g" event={"ID":"b6e49fe4-ed80-4c3a-b73a-feb001215541","Type":"ContainerDied","Data":"f75208e307966de6efed3c31dd842fc67ed8810aadc7579d0859a46a11b43b0f"} Jul 14 11:08:30.983876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.983806 2581 scope.go:117] "RemoveContainer" containerID="d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62" Jul 14 11:08:30.992552 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.992532 2581 scope.go:117] "RemoveContainer" containerID="d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62" Jul 14 11:08:30.992856 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:08:30.992833 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62\": container with ID starting with d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62 not found: ID does not exist" containerID="d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62" Jul 14 11:08:30.992937 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:30.992867 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62"} err="failed to get container status \"d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62\": rpc error: code = NotFound desc = could not find container \"d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62\": container with ID starting with d922910abe0a78efb29242cd15695f6ba816fc59459ca288dc83a31e91615a62 not found: ID does not exist" Jul 14 11:08:31.006991 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:31.006964 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8657c45967-npz6g"] Jul 14 11:08:31.014214 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:31.014190 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8657c45967-npz6g"] Jul 14 11:08:31.491339 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:31.491307 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e49fe4-ed80-4c3a-b73a-feb001215541" path="/var/lib/kubelet/pods/b6e49fe4-ed80-4c3a-b73a-feb001215541/volumes" Jul 14 11:08:33.345023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.344976 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f94f6c8cd-m6vzs"] Jul 14 11:08:33.373420 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.373394 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54d8f8f579-8zxk4"] Jul 14 11:08:33.373688 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.373676 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e49fe4-ed80-4c3a-b73a-feb001215541" containerName="registry" Jul 14 11:08:33.373733 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.373690 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e49fe4-ed80-4c3a-b73a-feb001215541" containerName="registry" Jul 14 11:08:33.373789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.373744 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e49fe4-ed80-4c3a-b73a-feb001215541" containerName="registry" Jul 14 11:08:33.377236 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.377191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.407510 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.407480 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54d8f8f579-8zxk4"] Jul 14 11:08:33.423995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.423971 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:33.476750 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.476722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-service-ca\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.476942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.476772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-oauth-serving-cert\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.476942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.476860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-oauth-config\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.476942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.476918 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-serving-cert\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.477083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.476942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-config\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.477083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.476971 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-trusted-ca-bundle\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.477083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.477036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2sk8\" (UniqueName: \"kubernetes.io/projected/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-kube-api-access-k2sk8\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578425 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-serving-cert\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578425 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-config\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-trusted-ca-bundle\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2sk8\" (UniqueName: \"kubernetes.io/projected/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-kube-api-access-k2sk8\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-service-ca\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-oauth-serving-cert\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.578684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.578684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-oauth-config\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.579371 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.579337 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-config\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.579489 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.579399 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-trusted-ca-bundle\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.579489 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.579426 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-service-ca\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.579489 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.579468 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-oauth-serving-cert\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.581023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.581002 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-serving-cert\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.581148 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.581054 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-oauth-config\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.585494 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.585470 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2sk8\" (UniqueName: \"kubernetes.io/projected/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-kube-api-access-k2sk8\") pod \"console-54d8f8f579-8zxk4\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.686083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.686053 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:33.819877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.817553 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54d8f8f579-8zxk4"] Jul 14 11:08:33.822981 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:33.822950 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19caed19_ab83_4d57_9b5a_8aaa3f417bbe.slice/crio-d36dd59cbc8c7005cd69721f5e3d6f43823a99d6f4e75f5e3649fb70513d6e29 WatchSource:0}: Error finding container d36dd59cbc8c7005cd69721f5e3d6f43823a99d6f4e75f5e3649fb70513d6e29: Status 404 returned error can't find the container with id d36dd59cbc8c7005cd69721f5e3d6f43823a99d6f4e75f5e3649fb70513d6e29 Jul 14 11:08:33.997954 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.997859 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d8f8f579-8zxk4" event={"ID":"19caed19-ab83-4d57-9b5a-8aaa3f417bbe","Type":"ContainerStarted","Data":"78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3"} Jul 14 11:08:33.997954 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:33.997918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d8f8f579-8zxk4" event={"ID":"19caed19-ab83-4d57-9b5a-8aaa3f417bbe","Type":"ContainerStarted","Data":"d36dd59cbc8c7005cd69721f5e3d6f43823a99d6f4e75f5e3649fb70513d6e29"} Jul 14 11:08:34.016970 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:34.016923 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54d8f8f579-8zxk4" podStartSLOduration=1.016908611 podStartE2EDuration="1.016908611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:08:34.016225119 +0000 UTC m=+104.999195954" watchObservedRunningTime="2026-07-14 11:08:34.016908611 +0000 UTC m=+104.999879444" Jul 14 11:08:42.994218 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:42.994184 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:42.994604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:42.994239 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:08:43.686267 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:43.686234 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:43.686455 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:43.686430 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:43.690999 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:43.690974 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:44.038595 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:44.038521 2581 generic.go:356] "Generic (PLEG): container finished" podID="f9d63cc1-acc6-435e-8f08-70a84ed537a5" containerID="627d435ad2fdec9a631658e2e91475f63daa0e2fd7c97f0de28aced0ccd0fd8d" exitCode=0 Jul 14 11:08:44.039000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:44.038595 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" event={"ID":"f9d63cc1-acc6-435e-8f08-70a84ed537a5","Type":"ContainerDied","Data":"627d435ad2fdec9a631658e2e91475f63daa0e2fd7c97f0de28aced0ccd0fd8d"} Jul 14 11:08:44.039000 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:44.038976 2581 scope.go:117] "RemoveContainer" containerID="627d435ad2fdec9a631658e2e91475f63daa0e2fd7c97f0de28aced0ccd0fd8d" Jul 14 11:08:44.042878 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:44.042860 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:08:44.108249 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:44.108220 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8cf5f657-2k4ts"] Jul 14 11:08:45.044403 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:45.044372 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-9w9hg" event={"ID":"f9d63cc1-acc6-435e-8f08-70a84ed537a5","Type":"ContainerStarted","Data":"2e2e4c6c5a52441e8f713c0860b9b980579227b48513deb06821c9ce2d06ab53"} Jul 14 11:08:58.366836 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.366784 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f94f6c8cd-m6vzs" podUID="12867806-1ad5-4eee-ac76-66ab1079d987" containerName="console" containerID="cri-o://6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a" gracePeriod=15 Jul 14 11:08:58.603896 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.603875 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:58.687672 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687649 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-oauth-config\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.687823 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687722 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-oauth-serving-cert\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.687863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687817 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-console-config\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.687863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687844 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-service-ca\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.687926 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687888 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-serving-cert\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.687926 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687912 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cswwx\" (UniqueName: \"kubernetes.io/projected/12867806-1ad5-4eee-ac76-66ab1079d987-kube-api-access-cswwx\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.688011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.687959 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-trusted-ca-bundle\") pod \"12867806-1ad5-4eee-ac76-66ab1079d987\" (UID: \"12867806-1ad5-4eee-ac76-66ab1079d987\") " Jul 14 11:08:58.688317 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.688185 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:08:58.688317 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.688229 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-console-config" (OuterVolumeSpecName: "console-config") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:08:58.688317 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.688254 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-service-ca" (OuterVolumeSpecName: "service-ca") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:08:58.688317 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.688292 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-oauth-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:58.688557 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.688451 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:08:58.689756 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.689733 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:08:58.689936 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.689918 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12867806-1ad5-4eee-ac76-66ab1079d987-kube-api-access-cswwx" (OuterVolumeSpecName: "kube-api-access-cswwx") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "kube-api-access-cswwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:08:58.689999 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.689929 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "12867806-1ad5-4eee-ac76-66ab1079d987" (UID: "12867806-1ad5-4eee-ac76-66ab1079d987"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:08:58.789323 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.789292 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:58.789323 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.789318 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cswwx\" (UniqueName: \"kubernetes.io/projected/12867806-1ad5-4eee-ac76-66ab1079d987-kube-api-access-cswwx\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:58.789323 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.789327 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-trusted-ca-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:58.789533 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.789335 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12867806-1ad5-4eee-ac76-66ab1079d987-console-oauth-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:58.789533 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.789345 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-console-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:58.789533 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:58.789353 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12867806-1ad5-4eee-ac76-66ab1079d987-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:08:59.099316 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.099230 2581 generic.go:356] "Generic (PLEG): container finished" podID="12867806-1ad5-4eee-ac76-66ab1079d987" containerID="6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a" exitCode=0 Jul 14 11:08:59.099316 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.099267 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94f6c8cd-m6vzs" event={"ID":"12867806-1ad5-4eee-ac76-66ab1079d987","Type":"ContainerDied","Data":"6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a"} Jul 14 11:08:59.099316 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.099287 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94f6c8cd-m6vzs" Jul 14 11:08:59.099531 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.099340 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94f6c8cd-m6vzs" event={"ID":"12867806-1ad5-4eee-ac76-66ab1079d987","Type":"ContainerDied","Data":"98a43ad4ec9d09e9d3f83c69d05804232c6b8eab6b0fdc0b39c4f1b43a2c2fe8"} Jul 14 11:08:59.099531 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.099361 2581 scope.go:117] "RemoveContainer" containerID="6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a" Jul 14 11:08:59.108037 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.108019 2581 scope.go:117] "RemoveContainer" containerID="6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a" Jul 14 11:08:59.108307 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:08:59.108289 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a\": container with ID starting with 6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a not found: ID does not exist" containerID="6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a" Jul 14 11:08:59.108359 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.108316 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a"} err="failed to get container status \"6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a\": rpc error: code = NotFound desc = could not find container \"6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a\": container with ID starting with 6b77f65a72f8fe309b858c12450b408e99c330a91f5e68ff2b468083bd12710a not found: ID does not exist" Jul 14 11:08:59.126894 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.126867 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f94f6c8cd-m6vzs"] Jul 14 11:08:59.136754 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.136735 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f94f6c8cd-m6vzs"] Jul 14 11:08:59.297146 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.297096 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:08:59.299276 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.299259 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1b73a1-619f-41c5-8a62-4380a23919cf-metrics-certs\") pod \"network-metrics-daemon-tp4pg\" (UID: \"4e1b73a1-619f-41c5-8a62-4380a23919cf\") " pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:08:59.312413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.312394 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhbfh\"" Jul 14 11:08:59.318662 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.318650 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tp4pg" Jul 14 11:08:59.450688 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.450657 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tp4pg"] Jul 14 11:08:59.453726 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:08:59.453690 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1b73a1_619f_41c5_8a62_4380a23919cf.slice/crio-5df9deea857d6a8cd70d692a7a7fbf32c7e5b1f8cde02717641c58f199e7b4b0 WatchSource:0}: Error finding container 5df9deea857d6a8cd70d692a7a7fbf32c7e5b1f8cde02717641c58f199e7b4b0: Status 404 returned error can't find the container with id 5df9deea857d6a8cd70d692a7a7fbf32c7e5b1f8cde02717641c58f199e7b4b0 Jul 14 11:08:59.490057 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:08:59.490023 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12867806-1ad5-4eee-ac76-66ab1079d987" path="/var/lib/kubelet/pods/12867806-1ad5-4eee-ac76-66ab1079d987/volumes" Jul 14 11:09:00.105188 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:00.105146 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tp4pg" event={"ID":"4e1b73a1-619f-41c5-8a62-4380a23919cf","Type":"ContainerStarted","Data":"5df9deea857d6a8cd70d692a7a7fbf32c7e5b1f8cde02717641c58f199e7b4b0"} Jul 14 11:09:01.111757 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:01.111723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tp4pg" event={"ID":"4e1b73a1-619f-41c5-8a62-4380a23919cf","Type":"ContainerStarted","Data":"57e78249c969d9b3b279561212b8c551fb2ec49b5f98e8202e147e02c08e0573"} Jul 14 11:09:01.112251 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:01.111770 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tp4pg" event={"ID":"4e1b73a1-619f-41c5-8a62-4380a23919cf","Type":"ContainerStarted","Data":"db6ccdeabf4fa795f7dbe4e31ef4fa4cab30858a8c653f7a45a851c2262383cc"} Jul 14 11:09:01.128750 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:01.128698 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tp4pg" podStartSLOduration=131.23418385 podStartE2EDuration="2m12.128676501s" totalImagesPullingTime="894.492651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:09:01.127931009 +0000 UTC m=+132.110901844" watchObservedRunningTime="2026-07-14 11:09:01.128676501 +0000 UTC m=+132.111647335" Jul 14 11:09:02.999556 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:02.999529 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:09:03.003408 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:03.003383 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-54885f4554-wt9dp" Jul 14 11:09:09.130462 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.130400 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d8cf5f657-2k4ts" podUID="b72681f5-580d-401b-a6e4-536ece78c8ae" containerName="console" containerID="cri-o://d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8" gracePeriod=15 Jul 14 11:09:09.394729 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.394678 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:09:09.481481 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.481426 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-oauth-serving-cert\") pod \"b72681f5-580d-401b-a6e4-536ece78c8ae\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " Jul 14 11:09:09.482293 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.482155 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b72681f5-580d-401b-a6e4-536ece78c8ae" (UID: "b72681f5-580d-401b-a6e4-536ece78c8ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:09:09.482868 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.482481 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-oauth-config\") pod \"b72681f5-580d-401b-a6e4-536ece78c8ae\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " Jul 14 11:09:09.483073 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.483036 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-serving-cert\") pod \"b72681f5-580d-401b-a6e4-536ece78c8ae\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " Jul 14 11:09:09.483859 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.483549 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhcgh\" (UniqueName: \"kubernetes.io/projected/b72681f5-580d-401b-a6e4-536ece78c8ae-kube-api-access-bhcgh\") pod \"b72681f5-580d-401b-a6e4-536ece78c8ae\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " Jul 14 11:09:09.484033 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.484018 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-console-config\") pod \"b72681f5-580d-401b-a6e4-536ece78c8ae\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " Jul 14 11:09:09.484220 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.484206 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-service-ca\") pod \"b72681f5-580d-401b-a6e4-536ece78c8ae\" (UID: \"b72681f5-580d-401b-a6e4-536ece78c8ae\") " Jul 14 11:09:09.484673 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.484657 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-oauth-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:09:09.485396 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.485348 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-console-config" (OuterVolumeSpecName: "console-config") pod "b72681f5-580d-401b-a6e4-536ece78c8ae" (UID: "b72681f5-580d-401b-a6e4-536ece78c8ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:09:09.485859 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.485828 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "b72681f5-580d-401b-a6e4-536ece78c8ae" (UID: "b72681f5-580d-401b-a6e4-536ece78c8ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:09:09.490884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.488494 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b72681f5-580d-401b-a6e4-536ece78c8ae" (UID: "b72681f5-580d-401b-a6e4-536ece78c8ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:09:09.490884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.490832 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72681f5-580d-401b-a6e4-536ece78c8ae-kube-api-access-bhcgh" (OuterVolumeSpecName: "kube-api-access-bhcgh") pod "b72681f5-580d-401b-a6e4-536ece78c8ae" (UID: "b72681f5-580d-401b-a6e4-536ece78c8ae"). InnerVolumeSpecName "kube-api-access-bhcgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:09:09.492475 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.492422 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b72681f5-580d-401b-a6e4-536ece78c8ae" (UID: "b72681f5-580d-401b-a6e4-536ece78c8ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:09:09.585470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.585443 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-oauth-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:09:09.585470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.585468 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72681f5-580d-401b-a6e4-536ece78c8ae-console-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:09:09.585470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.585478 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhcgh\" (UniqueName: \"kubernetes.io/projected/b72681f5-580d-401b-a6e4-536ece78c8ae-kube-api-access-bhcgh\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:09:09.585691 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.585487 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-console-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:09:09.585691 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:09.585495 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72681f5-580d-401b-a6e4-536ece78c8ae-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:09:10.148935 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.148894 2581 generic.go:356] "Generic (PLEG): container finished" podID="b72681f5-580d-401b-a6e4-536ece78c8ae" containerID="d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8" exitCode=0 Jul 14 11:09:10.149413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.148949 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8cf5f657-2k4ts" event={"ID":"b72681f5-580d-401b-a6e4-536ece78c8ae","Type":"ContainerDied","Data":"d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8"} Jul 14 11:09:10.149413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.148961 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8cf5f657-2k4ts" Jul 14 11:09:10.149413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.149000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8cf5f657-2k4ts" event={"ID":"b72681f5-580d-401b-a6e4-536ece78c8ae","Type":"ContainerDied","Data":"29d9e100f1bcd7ce81788a1b9e9acb7a345e3b543f090b1ce5aedc1af797b7a1"} Jul 14 11:09:10.149413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.149022 2581 scope.go:117] "RemoveContainer" containerID="d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8" Jul 14 11:09:10.157324 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.157302 2581 scope.go:117] "RemoveContainer" containerID="d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8" Jul 14 11:09:10.157575 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:09:10.157555 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8\": container with ID starting with d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8 not found: ID does not exist" containerID="d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8" Jul 14 11:09:10.157644 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.157588 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8"} err="failed to get container status \"d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8\": rpc error: code = NotFound desc = could not find container \"d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8\": container with ID starting with d62a914cf56f1471ec96f54a50d14d701ade4ed6653cc44cca8198a337ab1dc8 not found: ID does not exist" Jul 14 11:09:10.163909 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.163888 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8cf5f657-2k4ts"] Jul 14 11:09:10.165776 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:10.165756 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d8cf5f657-2k4ts"] Jul 14 11:09:11.491086 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:11.491053 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72681f5-580d-401b-a6e4-536ece78c8ae" path="/var/lib/kubelet/pods/b72681f5-580d-401b-a6e4-536ece78c8ae/volumes" Jul 14 11:09:57.937174 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937144 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-98885cdcb-wrqmn"] Jul 14 11:09:57.937653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937448 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72681f5-580d-401b-a6e4-536ece78c8ae" containerName="console" Jul 14 11:09:57.937653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937460 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72681f5-580d-401b-a6e4-536ece78c8ae" containerName="console" Jul 14 11:09:57.937653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937472 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12867806-1ad5-4eee-ac76-66ab1079d987" containerName="console" Jul 14 11:09:57.937653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937477 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12867806-1ad5-4eee-ac76-66ab1079d987" containerName="console" Jul 14 11:09:57.937653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937526 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72681f5-580d-401b-a6e4-536ece78c8ae" containerName="console" Jul 14 11:09:57.937653 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.937535 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12867806-1ad5-4eee-ac76-66ab1079d987" containerName="console" Jul 14 11:09:57.941140 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.941099 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:57.953843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:57.953821 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98885cdcb-wrqmn"] Jul 14 11:09:58.080988 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.080959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-trusted-ca-bundle\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.080988 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.080991 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lld7b\" (UniqueName: \"kubernetes.io/projected/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-kube-api-access-lld7b\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.081228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.081019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-oauth-serving-cert\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.081228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.081039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-serving-cert\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.081228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.081136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-oauth-config\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.081228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.081170 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-config\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.081228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.081226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-service-ca\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.181957 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.181920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-oauth-config\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.181957 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.181959 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-config\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182244 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.181978 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-service-ca\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182244 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-trusted-ca-bundle\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182244 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182063 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lld7b\" (UniqueName: \"kubernetes.io/projected/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-kube-api-access-lld7b\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182244 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182089 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-oauth-serving-cert\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182244 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182147 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-serving-cert\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182786 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182757 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-service-ca\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182881 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182849 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-oauth-serving-cert\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.182932 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.182891 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-config\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.183088 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.183065 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-trusted-ca-bundle\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.184539 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.184520 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-oauth-config\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.184630 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.184586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-serving-cert\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.190003 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.189954 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lld7b\" (UniqueName: \"kubernetes.io/projected/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-kube-api-access-lld7b\") pod \"console-98885cdcb-wrqmn\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.250312 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.250282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:09:58.382923 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:58.382889 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98885cdcb-wrqmn"] Jul 14 11:09:58.385844 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:09:58.385808 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a73c7d_17c0_4c3d_be0f_1efd45d9b6e4.slice/crio-62e9d8f0250a83f9474cfc6f863d4a382229aa67917470621ee1ca06e854b41b WatchSource:0}: Error finding container 62e9d8f0250a83f9474cfc6f863d4a382229aa67917470621ee1ca06e854b41b: Status 404 returned error can't find the container with id 62e9d8f0250a83f9474cfc6f863d4a382229aa67917470621ee1ca06e854b41b Jul 14 11:09:59.340701 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:59.340665 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98885cdcb-wrqmn" event={"ID":"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4","Type":"ContainerStarted","Data":"7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a"} Jul 14 11:09:59.341141 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:59.340716 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98885cdcb-wrqmn" event={"ID":"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4","Type":"ContainerStarted","Data":"62e9d8f0250a83f9474cfc6f863d4a382229aa67917470621ee1ca06e854b41b"} Jul 14 11:09:59.359947 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:09:59.359899 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-98885cdcb-wrqmn" podStartSLOduration=2.359880416 podStartE2EDuration="2.359880416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:09:59.358939491 +0000 UTC m=+190.341910326" watchObservedRunningTime="2026-07-14 11:09:59.359880416 +0000 UTC m=+190.342851253" Jul 14 11:10:08.250991 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:08.250959 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:10:08.251381 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:08.251014 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:10:08.255425 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:08.255406 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:10:08.375611 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:08.375586 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:10:08.423529 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:08.423505 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54d8f8f579-8zxk4"] Jul 14 11:10:33.449744 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.449708 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54d8f8f579-8zxk4" podUID="19caed19-ab83-4d57-9b5a-8aaa3f417bbe" containerName="console" containerID="cri-o://78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3" gracePeriod=15 Jul 14 11:10:33.688555 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.688533 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:10:33.764733 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764670 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-serving-cert\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.764733 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764717 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-config\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.764933 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764757 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-trusted-ca-bundle\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.764933 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764787 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-oauth-serving-cert\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.764933 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764816 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-service-ca\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.764933 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764843 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2sk8\" (UniqueName: \"kubernetes.io/projected/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-kube-api-access-k2sk8\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.764933 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.764876 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-oauth-config\") pod \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\" (UID: \"19caed19-ab83-4d57-9b5a-8aaa3f417bbe\") " Jul 14 11:10:33.765259 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.765227 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:10:33.765331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.765251 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:10:33.765331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.765257 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-config" (OuterVolumeSpecName: "console-config") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:10:33.765331 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.765297 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-service-ca" (OuterVolumeSpecName: "service-ca") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:10:33.766757 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.766729 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:10:33.766893 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.766874 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-kube-api-access-k2sk8" (OuterVolumeSpecName: "kube-api-access-k2sk8") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "kube-api-access-k2sk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:10:33.767025 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.767001 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "19caed19-ab83-4d57-9b5a-8aaa3f417bbe" (UID: "19caed19-ab83-4d57-9b5a-8aaa3f417bbe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:10:33.866341 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866321 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:33.866341 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866338 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2sk8\" (UniqueName: \"kubernetes.io/projected/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-kube-api-access-k2sk8\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:33.866458 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866348 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-oauth-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:33.866458 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866357 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:33.866458 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866375 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-console-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:33.866458 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866383 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-trusted-ca-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:33.866458 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:33.866390 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19caed19-ab83-4d57-9b5a-8aaa3f417bbe-oauth-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:10:34.467024 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.466987 2581 generic.go:356] "Generic (PLEG): container finished" podID="19caed19-ab83-4d57-9b5a-8aaa3f417bbe" containerID="78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3" exitCode=0 Jul 14 11:10:34.467024 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.467026 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d8f8f579-8zxk4" Jul 14 11:10:34.467445 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.467055 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d8f8f579-8zxk4" event={"ID":"19caed19-ab83-4d57-9b5a-8aaa3f417bbe","Type":"ContainerDied","Data":"78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3"} Jul 14 11:10:34.467445 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.467102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d8f8f579-8zxk4" event={"ID":"19caed19-ab83-4d57-9b5a-8aaa3f417bbe","Type":"ContainerDied","Data":"d36dd59cbc8c7005cd69721f5e3d6f43823a99d6f4e75f5e3649fb70513d6e29"} Jul 14 11:10:34.467445 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.467140 2581 scope.go:117] "RemoveContainer" containerID="78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3" Jul 14 11:10:34.475624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.475608 2581 scope.go:117] "RemoveContainer" containerID="78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3" Jul 14 11:10:34.475865 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:10:34.475844 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3\": container with ID starting with 78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3 not found: ID does not exist" containerID="78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3" Jul 14 11:10:34.475910 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.475874 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3"} err="failed to get container status \"78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3\": rpc error: code = NotFound desc = could not find container \"78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3\": container with ID starting with 78ceb63e7f3eef36885da5eff4966c704ee0ae7f7a1204e3fa70c78442d089e3 not found: ID does not exist" Jul 14 11:10:34.488395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.488376 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54d8f8f579-8zxk4"] Jul 14 11:10:34.492939 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:34.492914 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54d8f8f579-8zxk4"] Jul 14 11:10:35.490656 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:35.490622 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19caed19-ab83-4d57-9b5a-8aaa3f417bbe" path="/var/lib/kubelet/pods/19caed19-ab83-4d57-9b5a-8aaa3f417bbe/volumes" Jul 14 11:10:42.326577 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.326548 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc"] Jul 14 11:10:42.326950 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.326846 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19caed19-ab83-4d57-9b5a-8aaa3f417bbe" containerName="console" Jul 14 11:10:42.326950 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.326860 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="19caed19-ab83-4d57-9b5a-8aaa3f417bbe" containerName="console" Jul 14 11:10:42.326950 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.326919 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="19caed19-ab83-4d57-9b5a-8aaa3f417bbe" containerName="console" Jul 14 11:10:42.329578 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.329560 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.331842 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.331822 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vlc9t\"" Jul 14 11:10:42.332156 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.332143 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 11:10:42.332563 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.332546 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 11:10:42.342468 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.342449 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc"] Jul 14 11:10:42.424050 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.424028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.424197 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.424054 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.424197 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.424104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rplpv\" (UniqueName: \"kubernetes.io/projected/38559387-9b8c-4e37-b17d-d38be6dd7d38-kube-api-access-rplpv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.524812 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.524787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rplpv\" (UniqueName: \"kubernetes.io/projected/38559387-9b8c-4e37-b17d-d38be6dd7d38-kube-api-access-rplpv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.524981 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.524840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.524981 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.524858 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.525221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.525202 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.525287 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.525261 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.533238 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.533219 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rplpv\" (UniqueName: \"kubernetes.io/projected/38559387-9b8c-4e37-b17d-d38be6dd7d38-kube-api-access-rplpv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.638349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.638311 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:10:42.767658 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:42.767531 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc"] Jul 14 11:10:42.770358 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:10:42.770327 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38559387_9b8c_4e37_b17d_d38be6dd7d38.slice/crio-bdf3841f658f0ce682208f19cf4ee319c9e1d4b0f54c58d28e2cbd9a7a572769 WatchSource:0}: Error finding container bdf3841f658f0ce682208f19cf4ee319c9e1d4b0f54c58d28e2cbd9a7a572769: Status 404 returned error can't find the container with id bdf3841f658f0ce682208f19cf4ee319c9e1d4b0f54c58d28e2cbd9a7a572769 Jul 14 11:10:43.500710 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:43.500676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" event={"ID":"38559387-9b8c-4e37-b17d-d38be6dd7d38","Type":"ContainerStarted","Data":"bdf3841f658f0ce682208f19cf4ee319c9e1d4b0f54c58d28e2cbd9a7a572769"} Jul 14 11:10:49.525095 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:49.525061 2581 generic.go:356] "Generic (PLEG): container finished" podID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerID="f3e4838b932d25fd00fcd13b08311aa65ac18bdb9e1f10b258d19c992d6b853f" exitCode=0 Jul 14 11:10:49.525443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:49.525145 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" event={"ID":"38559387-9b8c-4e37-b17d-d38be6dd7d38","Type":"ContainerDied","Data":"f3e4838b932d25fd00fcd13b08311aa65ac18bdb9e1f10b258d19c992d6b853f"} Jul 14 11:10:51.536296 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:51.536266 2581 generic.go:356] "Generic (PLEG): container finished" podID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerID="e3e933ecdad007216953ffb5bdd89739574481ed1ac98e0e38726ff988515ad4" exitCode=0 Jul 14 11:10:51.536605 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:51.536301 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" event={"ID":"38559387-9b8c-4e37-b17d-d38be6dd7d38","Type":"ContainerDied","Data":"e3e933ecdad007216953ffb5bdd89739574481ed1ac98e0e38726ff988515ad4"} Jul 14 11:10:59.574101 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:59.574072 2581 generic.go:356] "Generic (PLEG): container finished" podID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerID="458ac666c4a37fb11cc60902204ed6019378cc40bc462dab693c890d97c245b3" exitCode=0 Jul 14 11:10:59.574474 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:10:59.574140 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" event={"ID":"38559387-9b8c-4e37-b17d-d38be6dd7d38","Type":"ContainerDied","Data":"458ac666c4a37fb11cc60902204ed6019378cc40bc462dab693c890d97c245b3"} Jul 14 11:11:00.703251 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.703227 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:11:00.880693 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.880658 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-util\") pod \"38559387-9b8c-4e37-b17d-d38be6dd7d38\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " Jul 14 11:11:00.880855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.880703 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rplpv\" (UniqueName: \"kubernetes.io/projected/38559387-9b8c-4e37-b17d-d38be6dd7d38-kube-api-access-rplpv\") pod \"38559387-9b8c-4e37-b17d-d38be6dd7d38\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " Jul 14 11:11:00.880855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.880734 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-bundle\") pod \"38559387-9b8c-4e37-b17d-d38be6dd7d38\" (UID: \"38559387-9b8c-4e37-b17d-d38be6dd7d38\") " Jul 14 11:11:00.881442 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.881415 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-bundle" (OuterVolumeSpecName: "bundle") pod "38559387-9b8c-4e37-b17d-d38be6dd7d38" (UID: "38559387-9b8c-4e37-b17d-d38be6dd7d38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:11:00.882891 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.882860 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38559387-9b8c-4e37-b17d-d38be6dd7d38-kube-api-access-rplpv" (OuterVolumeSpecName: "kube-api-access-rplpv") pod "38559387-9b8c-4e37-b17d-d38be6dd7d38" (UID: "38559387-9b8c-4e37-b17d-d38be6dd7d38"). InnerVolumeSpecName "kube-api-access-rplpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:11:00.884763 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.884737 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-util" (OuterVolumeSpecName: "util") pod "38559387-9b8c-4e37-b17d-d38be6dd7d38" (UID: "38559387-9b8c-4e37-b17d-d38be6dd7d38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:11:00.981951 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.981926 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:00.981951 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.981949 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rplpv\" (UniqueName: \"kubernetes.io/projected/38559387-9b8c-4e37-b17d-d38be6dd7d38-kube-api-access-rplpv\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:00.982089 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:00.981960 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38559387-9b8c-4e37-b17d-d38be6dd7d38-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:01.583840 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:01.583808 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" event={"ID":"38559387-9b8c-4e37-b17d-d38be6dd7d38","Type":"ContainerDied","Data":"bdf3841f658f0ce682208f19cf4ee319c9e1d4b0f54c58d28e2cbd9a7a572769"} Jul 14 11:11:01.583978 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:01.583850 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sz5mc" Jul 14 11:11:01.583978 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:01.583875 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdf3841f658f0ce682208f19cf4ee319c9e1d4b0f54c58d28e2cbd9a7a572769" Jul 14 11:11:10.860670 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860635 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722"] Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860919 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="pull" Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860929 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="pull" Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860938 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="extract" Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860944 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="extract" Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860962 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="util" Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.860967 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="util" Jul 14 11:11:10.861178 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.861013 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="38559387-9b8c-4e37-b17d-d38be6dd7d38" containerName="extract" Jul 14 11:11:10.872282 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.872236 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:10.874945 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.874919 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vlc9t\"" Jul 14 11:11:10.875797 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.875777 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 11:11:10.877682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.877661 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 11:11:10.903040 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:10.903016 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722"] Jul 14 11:11:11.057652 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.057625 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.057811 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.057659 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/bb598c38-1b6a-4bae-ad40-097da0e0015d-kube-api-access-h9dp6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.057811 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.057712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.158424 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.158401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.158544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.158447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.158544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.158472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/bb598c38-1b6a-4bae-ad40-097da0e0015d-kube-api-access-h9dp6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.158767 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.158751 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.158816 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.158801 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.218874 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.218846 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/bb598c38-1b6a-4bae-ad40-097da0e0015d-kube-api-access-h9dp6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.481261 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.481198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:11.672640 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:11:11.672613 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb598c38_1b6a_4bae_ad40_097da0e0015d.slice/crio-6db920d51624f658f4964840cdff04102972cdc790094354c254e6f96370c9c4 WatchSource:0}: Error finding container 6db920d51624f658f4964840cdff04102972cdc790094354c254e6f96370c9c4: Status 404 returned error can't find the container with id 6db920d51624f658f4964840cdff04102972cdc790094354c254e6f96370c9c4 Jul 14 11:11:11.675075 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:11.675039 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722"] Jul 14 11:11:12.624049 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.624015 2581 generic.go:356] "Generic (PLEG): container finished" podID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerID="7d144bb3236cd8113dafc2dbf656ec0c4e12e7c041edb390868647d9c48ee8c6" exitCode=0 Jul 14 11:11:12.624451 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.624098 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" event={"ID":"bb598c38-1b6a-4bae-ad40-097da0e0015d","Type":"ContainerDied","Data":"7d144bb3236cd8113dafc2dbf656ec0c4e12e7c041edb390868647d9c48ee8c6"} Jul 14 11:11:12.624451 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.624155 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" event={"ID":"bb598c38-1b6a-4bae-ad40-097da0e0015d","Type":"ContainerStarted","Data":"6db920d51624f658f4964840cdff04102972cdc790094354c254e6f96370c9c4"} Jul 14 11:11:12.988824 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.988748 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6lwth"] Jul 14 11:11:12.990858 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.990842 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:12.992845 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.992822 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 11:11:12.992845 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.992837 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 11:11:12.993309 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:12.993293 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jjmxl\"" Jul 14 11:11:13.001350 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.001332 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6lwth"] Jul 14 11:11:13.174431 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.174402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8zfr\" (UniqueName: \"kubernetes.io/projected/4ebf9548-b421-4efa-b753-a7b41f150784-kube-api-access-q8zfr\") pod \"cert-manager-cainjector-559cdd8c87-6lwth\" (UID: \"4ebf9548-b421-4efa-b753-a7b41f150784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.174588 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.174439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ebf9548-b421-4efa-b753-a7b41f150784-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6lwth\" (UID: \"4ebf9548-b421-4efa-b753-a7b41f150784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.275684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.275616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8zfr\" (UniqueName: \"kubernetes.io/projected/4ebf9548-b421-4efa-b753-a7b41f150784-kube-api-access-q8zfr\") pod \"cert-manager-cainjector-559cdd8c87-6lwth\" (UID: \"4ebf9548-b421-4efa-b753-a7b41f150784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.275684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.275655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ebf9548-b421-4efa-b753-a7b41f150784-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6lwth\" (UID: \"4ebf9548-b421-4efa-b753-a7b41f150784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.287604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.287581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ebf9548-b421-4efa-b753-a7b41f150784-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6lwth\" (UID: \"4ebf9548-b421-4efa-b753-a7b41f150784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.287743 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.287723 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8zfr\" (UniqueName: \"kubernetes.io/projected/4ebf9548-b421-4efa-b753-a7b41f150784-kube-api-access-q8zfr\") pod \"cert-manager-cainjector-559cdd8c87-6lwth\" (UID: \"4ebf9548-b421-4efa-b753-a7b41f150784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.300488 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.300467 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" Jul 14 11:11:13.426942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.426790 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6lwth"] Jul 14 11:11:13.429422 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:11:13.429382 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ebf9548_b421_4efa_b753_a7b41f150784.slice/crio-c2461e073a4422e332bc3d53fc7b855734b442a668f6d33c2cf3dd14030f730d WatchSource:0}: Error finding container c2461e073a4422e332bc3d53fc7b855734b442a668f6d33c2cf3dd14030f730d: Status 404 returned error can't find the container with id c2461e073a4422e332bc3d53fc7b855734b442a668f6d33c2cf3dd14030f730d Jul 14 11:11:13.629424 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:13.629389 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" event={"ID":"4ebf9548-b421-4efa-b753-a7b41f150784","Type":"ContainerStarted","Data":"c2461e073a4422e332bc3d53fc7b855734b442a668f6d33c2cf3dd14030f730d"} Jul 14 11:11:15.641754 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:15.641721 2581 generic.go:356] "Generic (PLEG): container finished" podID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerID="ee99d9297b7d5bc30139c77ce84f62bcf8b380756e67d7c5a035fcecb6880ae6" exitCode=0 Jul 14 11:11:15.642174 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:15.641811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" event={"ID":"bb598c38-1b6a-4bae-ad40-097da0e0015d","Type":"ContainerDied","Data":"ee99d9297b7d5bc30139c77ce84f62bcf8b380756e67d7c5a035fcecb6880ae6"} Jul 14 11:11:16.647404 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:16.647369 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" event={"ID":"4ebf9548-b421-4efa-b753-a7b41f150784","Type":"ContainerStarted","Data":"19cbec152c2973722cd1b4651d34c637b677af1ce43cbaa23598ce6783285bf9"} Jul 14 11:11:16.649311 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:16.649289 2581 generic.go:356] "Generic (PLEG): container finished" podID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerID="288756d6351b783d719b42544729203cd120e413b73d617db548154484b9a923" exitCode=0 Jul 14 11:11:16.649408 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:16.649341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" event={"ID":"bb598c38-1b6a-4bae-ad40-097da0e0015d","Type":"ContainerDied","Data":"288756d6351b783d719b42544729203cd120e413b73d617db548154484b9a923"} Jul 14 11:11:16.668329 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:16.668286 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6lwth" podStartSLOduration=2.03244061 podStartE2EDuration="4.668271939s" totalImagesPullingTime="2.635831329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:16.666760368 +0000 UTC m=+267.649731202" watchObservedRunningTime="2026-07-14 11:11:16.668271939 +0000 UTC m=+267.651242774" Jul 14 11:11:17.774428 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.774407 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:17.812491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.812465 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/bb598c38-1b6a-4bae-ad40-097da0e0015d-kube-api-access-h9dp6\") pod \"bb598c38-1b6a-4bae-ad40-097da0e0015d\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " Jul 14 11:11:17.812626 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.812537 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-util\") pod \"bb598c38-1b6a-4bae-ad40-097da0e0015d\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " Jul 14 11:11:17.812626 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.812564 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-bundle\") pod \"bb598c38-1b6a-4bae-ad40-097da0e0015d\" (UID: \"bb598c38-1b6a-4bae-ad40-097da0e0015d\") " Jul 14 11:11:17.812973 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.812935 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-bundle" (OuterVolumeSpecName: "bundle") pod "bb598c38-1b6a-4bae-ad40-097da0e0015d" (UID: "bb598c38-1b6a-4bae-ad40-097da0e0015d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:11:17.814418 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.814396 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb598c38-1b6a-4bae-ad40-097da0e0015d-kube-api-access-h9dp6" (OuterVolumeSpecName: "kube-api-access-h9dp6") pod "bb598c38-1b6a-4bae-ad40-097da0e0015d" (UID: "bb598c38-1b6a-4bae-ad40-097da0e0015d"). InnerVolumeSpecName "kube-api-access-h9dp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:11:17.820294 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.820269 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-util" (OuterVolumeSpecName: "util") pod "bb598c38-1b6a-4bae-ad40-097da0e0015d" (UID: "bb598c38-1b6a-4bae-ad40-097da0e0015d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:11:17.913987 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.913909 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/bb598c38-1b6a-4bae-ad40-097da0e0015d-kube-api-access-h9dp6\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:17.913987 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.913946 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:17.913987 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:17.913957 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb598c38-1b6a-4bae-ad40-097da0e0015d-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:18.658242 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:18.658215 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" Jul 14 11:11:18.658397 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:18.658204 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbx722" event={"ID":"bb598c38-1b6a-4bae-ad40-097da0e0015d","Type":"ContainerDied","Data":"6db920d51624f658f4964840cdff04102972cdc790094354c254e6f96370c9c4"} Jul 14 11:11:18.658397 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:18.658338 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6db920d51624f658f4964840cdff04102972cdc790094354c254e6f96370c9c4" Jul 14 11:11:27.206032 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206001 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9"] Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206295 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="util" Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206307 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="util" Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206320 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="pull" Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206325 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="pull" Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206337 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="extract" Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206342 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="extract" Jul 14 11:11:27.206427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.206384 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb598c38-1b6a-4bae-ad40-097da0e0015d" containerName="extract" Jul 14 11:11:27.208235 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.208218 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.210365 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.210342 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5gttz\"" Jul 14 11:11:27.210747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.210729 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:11:27.211022 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.211004 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 11:11:27.218048 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.218028 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9"] Jul 14 11:11:27.285807 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.285786 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pj2p\" (UniqueName: \"kubernetes.io/projected/ccf52dba-e59f-430b-9ff6-8618c1c2121e-kube-api-access-5pj2p\") pod \"openshift-lws-operator-fd8ccff4c-lz6j9\" (UID: \"ccf52dba-e59f-430b-9ff6-8618c1c2121e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.285911 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.285826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccf52dba-e59f-430b-9ff6-8618c1c2121e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lz6j9\" (UID: \"ccf52dba-e59f-430b-9ff6-8618c1c2121e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.386354 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.386324 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pj2p\" (UniqueName: \"kubernetes.io/projected/ccf52dba-e59f-430b-9ff6-8618c1c2121e-kube-api-access-5pj2p\") pod \"openshift-lws-operator-fd8ccff4c-lz6j9\" (UID: \"ccf52dba-e59f-430b-9ff6-8618c1c2121e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.386463 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.386370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccf52dba-e59f-430b-9ff6-8618c1c2121e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lz6j9\" (UID: \"ccf52dba-e59f-430b-9ff6-8618c1c2121e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.386657 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.386642 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccf52dba-e59f-430b-9ff6-8618c1c2121e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lz6j9\" (UID: \"ccf52dba-e59f-430b-9ff6-8618c1c2121e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.394859 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.394834 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pj2p\" (UniqueName: \"kubernetes.io/projected/ccf52dba-e59f-430b-9ff6-8618c1c2121e-kube-api-access-5pj2p\") pod \"openshift-lws-operator-fd8ccff4c-lz6j9\" (UID: \"ccf52dba-e59f-430b-9ff6-8618c1c2121e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.517546 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.517471 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" Jul 14 11:11:27.647342 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.647315 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9"] Jul 14 11:11:27.650600 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:11:27.650573 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccf52dba_e59f_430b_9ff6_8618c1c2121e.slice/crio-b65618bf29334795c9480bf079aa95898583e7642e5c0f904d5f15e3d7a12775 WatchSource:0}: Error finding container b65618bf29334795c9480bf079aa95898583e7642e5c0f904d5f15e3d7a12775: Status 404 returned error can't find the container with id b65618bf29334795c9480bf079aa95898583e7642e5c0f904d5f15e3d7a12775 Jul 14 11:11:27.693057 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:27.693013 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" event={"ID":"ccf52dba-e59f-430b-9ff6-8618c1c2121e","Type":"ContainerStarted","Data":"b65618bf29334795c9480bf079aa95898583e7642e5c0f904d5f15e3d7a12775"} Jul 14 11:11:30.707902 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:30.707868 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" event={"ID":"ccf52dba-e59f-430b-9ff6-8618c1c2121e","Type":"ContainerStarted","Data":"b572e3efa1cd8f178dcdadb65bee288714f31d563d3bafda0fba6dbe9287adc1"} Jul 14 11:11:34.299533 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.299471 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lz6j9" podStartSLOduration=4.969246641 podStartE2EDuration="7.29945237s" totalImagesPullingTime="2.330205729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:30.734468546 +0000 UTC m=+281.717439382" watchObservedRunningTime="2026-07-14 11:11:34.29945237 +0000 UTC m=+285.282423206" Jul 14 11:11:34.300825 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.300803 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl"] Jul 14 11:11:34.303153 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.303139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.305320 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.305298 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 11:11:34.306004 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.305988 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 11:11:34.306085 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.306041 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vlc9t\"" Jul 14 11:11:34.316828 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.316808 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl"] Jul 14 11:11:34.339388 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.339370 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.339481 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.339396 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.339481 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.339417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg4w5\" (UniqueName: \"kubernetes.io/projected/ca7c9276-6716-477f-9d47-ca7c35b74189-kube-api-access-wg4w5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.440348 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.440323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.440436 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.440354 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg4w5\" (UniqueName: \"kubernetes.io/projected/ca7c9276-6716-477f-9d47-ca7c35b74189-kube-api-access-wg4w5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.440436 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.440419 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.440672 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.440655 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.440714 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.440682 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.451763 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.451732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg4w5\" (UniqueName: \"kubernetes.io/projected/ca7c9276-6716-477f-9d47-ca7c35b74189-kube-api-access-wg4w5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.612259 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.612206 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:34.741841 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:34.741814 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl"] Jul 14 11:11:34.744397 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:11:34.744367 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7c9276_6716_477f_9d47_ca7c35b74189.slice/crio-d0b0d2ef367be3e8c2f40ab81dcc1288c4cb1bd0cca1a67d0dfb67aca2fe0097 WatchSource:0}: Error finding container d0b0d2ef367be3e8c2f40ab81dcc1288c4cb1bd0cca1a67d0dfb67aca2fe0097: Status 404 returned error can't find the container with id d0b0d2ef367be3e8c2f40ab81dcc1288c4cb1bd0cca1a67d0dfb67aca2fe0097 Jul 14 11:11:35.728872 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:35.728793 2581 generic.go:356] "Generic (PLEG): container finished" podID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerID="a59cc21b86706f477348a6703d6aa426dde23c1165bd5b11a5a3cb4a6af752ed" exitCode=0 Jul 14 11:11:35.728872 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:35.728862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" event={"ID":"ca7c9276-6716-477f-9d47-ca7c35b74189","Type":"ContainerDied","Data":"a59cc21b86706f477348a6703d6aa426dde23c1165bd5b11a5a3cb4a6af752ed"} Jul 14 11:11:35.729271 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:35.728898 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" event={"ID":"ca7c9276-6716-477f-9d47-ca7c35b74189","Type":"ContainerStarted","Data":"d0b0d2ef367be3e8c2f40ab81dcc1288c4cb1bd0cca1a67d0dfb67aca2fe0097"} Jul 14 11:11:36.735125 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:36.735083 2581 generic.go:356] "Generic (PLEG): container finished" podID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerID="2e140a184f3a8d6f121d0f9561d0950e20e91cd48e674019a8ecec72c6c7a5b2" exitCode=0 Jul 14 11:11:36.735481 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:36.735173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" event={"ID":"ca7c9276-6716-477f-9d47-ca7c35b74189","Type":"ContainerDied","Data":"2e140a184f3a8d6f121d0f9561d0950e20e91cd48e674019a8ecec72c6c7a5b2"} Jul 14 11:11:37.741551 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:37.741520 2581 generic.go:356] "Generic (PLEG): container finished" podID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerID="55efb3430d40cb9efcc1236b1df5fd78602f9eede77077ef68c7ad6797e6893c" exitCode=0 Jul 14 11:11:37.741952 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:37.741587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" event={"ID":"ca7c9276-6716-477f-9d47-ca7c35b74189","Type":"ContainerDied","Data":"55efb3430d40cb9efcc1236b1df5fd78602f9eede77077ef68c7ad6797e6893c"} Jul 14 11:11:38.868044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.868021 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:38.977747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.977722 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-bundle\") pod \"ca7c9276-6716-477f-9d47-ca7c35b74189\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " Jul 14 11:11:38.977747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.977750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-util\") pod \"ca7c9276-6716-477f-9d47-ca7c35b74189\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " Jul 14 11:11:38.977919 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.977771 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg4w5\" (UniqueName: \"kubernetes.io/projected/ca7c9276-6716-477f-9d47-ca7c35b74189-kube-api-access-wg4w5\") pod \"ca7c9276-6716-477f-9d47-ca7c35b74189\" (UID: \"ca7c9276-6716-477f-9d47-ca7c35b74189\") " Jul 14 11:11:38.978435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.978412 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-bundle" (OuterVolumeSpecName: "bundle") pod "ca7c9276-6716-477f-9d47-ca7c35b74189" (UID: "ca7c9276-6716-477f-9d47-ca7c35b74189"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:11:38.979807 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.979787 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca7c9276-6716-477f-9d47-ca7c35b74189-kube-api-access-wg4w5" (OuterVolumeSpecName: "kube-api-access-wg4w5") pod "ca7c9276-6716-477f-9d47-ca7c35b74189" (UID: "ca7c9276-6716-477f-9d47-ca7c35b74189"). InnerVolumeSpecName "kube-api-access-wg4w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:11:38.983557 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:38.983537 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-util" (OuterVolumeSpecName: "util") pod "ca7c9276-6716-477f-9d47-ca7c35b74189" (UID: "ca7c9276-6716-477f-9d47-ca7c35b74189"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:11:39.079250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:39.079193 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:39.079250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:39.079213 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca7c9276-6716-477f-9d47-ca7c35b74189-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:39.079250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:39.079222 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg4w5\" (UniqueName: \"kubernetes.io/projected/ca7c9276-6716-477f-9d47-ca7c35b74189-kube-api-access-wg4w5\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:11:39.750861 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:39.750790 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" Jul 14 11:11:39.750992 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:39.750795 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd2svl" event={"ID":"ca7c9276-6716-477f-9d47-ca7c35b74189","Type":"ContainerDied","Data":"d0b0d2ef367be3e8c2f40ab81dcc1288c4cb1bd0cca1a67d0dfb67aca2fe0097"} Jul 14 11:11:39.750992 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:39.750906 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0b0d2ef367be3e8c2f40ab81dcc1288c4cb1bd0cca1a67d0dfb67aca2fe0097" Jul 14 11:11:46.961320 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961292 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2"] Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961562 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="pull" Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961572 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="pull" Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961579 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="extract" Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961584 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="extract" Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961593 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="util" Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961598 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="util" Jul 14 11:11:46.961781 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.961654 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca7c9276-6716-477f-9d47-ca7c35b74189" containerName="extract" Jul 14 11:11:46.964662 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.964645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:46.968908 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.968888 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 11:11:46.969134 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.969100 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dzl9m\"" Jul 14 11:11:46.970480 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.970459 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 11:11:46.970619 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.970539 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 11:11:46.970619 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.970587 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 11:11:46.997193 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:46.997169 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2"] Jul 14 11:11:47.041486 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.041465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbe1276-2227-4a1e-9ba0-02a72d128d63-tmp\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.041606 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.041493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/abbe1276-2227-4a1e-9ba0-02a72d128d63-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.041606 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.041515 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abbe1276-2227-4a1e-9ba0-02a72d128d63-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.041606 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.041554 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/abbe1276-2227-4a1e-9ba0-02a72d128d63-manifests\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.041708 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.041621 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcsjl\" (UniqueName: \"kubernetes.io/projected/abbe1276-2227-4a1e-9ba0-02a72d128d63-kube-api-access-jcsjl\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.142934 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.142897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbe1276-2227-4a1e-9ba0-02a72d128d63-tmp\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.142934 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.142939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/abbe1276-2227-4a1e-9ba0-02a72d128d63-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.143096 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.142959 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abbe1276-2227-4a1e-9ba0-02a72d128d63-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.143096 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.142991 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/abbe1276-2227-4a1e-9ba0-02a72d128d63-manifests\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.143096 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.143017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcsjl\" (UniqueName: \"kubernetes.io/projected/abbe1276-2227-4a1e-9ba0-02a72d128d63-kube-api-access-jcsjl\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.143386 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.143369 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/abbe1276-2227-4a1e-9ba0-02a72d128d63-manifests\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.145258 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.145226 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbe1276-2227-4a1e-9ba0-02a72d128d63-tmp\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.145401 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.145386 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/abbe1276-2227-4a1e-9ba0-02a72d128d63-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.145443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.145424 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abbe1276-2227-4a1e-9ba0-02a72d128d63-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.210386 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.210363 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcsjl\" (UniqueName: \"kubernetes.io/projected/abbe1276-2227-4a1e-9ba0-02a72d128d63-kube-api-access-jcsjl\") pod \"opendatahub-operator-controller-manager-7ff488f585-vrcv2\" (UID: \"abbe1276-2227-4a1e-9ba0-02a72d128d63\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.274435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.274380 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:47.479682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.479649 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2"] Jul 14 11:11:47.483414 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:11:47.483384 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbe1276_2227_4a1e_9ba0_02a72d128d63.slice/crio-33e8d6f59ae85e1beb77ec2fe4a673ae19820fafe8f7bd449b17d1dfe90c0c3a WatchSource:0}: Error finding container 33e8d6f59ae85e1beb77ec2fe4a673ae19820fafe8f7bd449b17d1dfe90c0c3a: Status 404 returned error can't find the container with id 33e8d6f59ae85e1beb77ec2fe4a673ae19820fafe8f7bd449b17d1dfe90c0c3a Jul 14 11:11:47.785177 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:47.785141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" event={"ID":"abbe1276-2227-4a1e-9ba0-02a72d128d63","Type":"ContainerStarted","Data":"33e8d6f59ae85e1beb77ec2fe4a673ae19820fafe8f7bd449b17d1dfe90c0c3a"} Jul 14 11:11:50.484545 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:50.484519 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:11:50.485983 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:50.484664 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:11:50.487580 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:50.487560 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:11:50.487811 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:50.487795 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:11:50.494561 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:50.494543 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 11:11:50.800482 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:50.800406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" event={"ID":"abbe1276-2227-4a1e-9ba0-02a72d128d63","Type":"ContainerStarted","Data":"913a1053058e4af56cd97689e35e88ad8ac77f43ef297bf616ff5a5f9b50165d"} Jul 14 11:11:51.807162 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:51.807135 2581 generic.go:356] "Generic (PLEG): container finished" podID="abbe1276-2227-4a1e-9ba0-02a72d128d63" containerID="913a1053058e4af56cd97689e35e88ad8ac77f43ef297bf616ff5a5f9b50165d" exitCode=0 Jul 14 11:11:51.807518 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:51.807213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" event={"ID":"abbe1276-2227-4a1e-9ba0-02a72d128d63","Type":"ContainerDied","Data":"913a1053058e4af56cd97689e35e88ad8ac77f43ef297bf616ff5a5f9b50165d"} Jul 14 11:11:51.807707 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:51.807693 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:11:52.815499 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:52.815406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" event={"ID":"abbe1276-2227-4a1e-9ba0-02a72d128d63","Type":"ContainerStarted","Data":"03fe4a10cf6353c2ca517173bdc49ec19e1e86b3e52c79c344e24e3c8f266448"} Jul 14 11:11:52.815886 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:52.815565 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:11:52.883963 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:11:52.883912 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" podStartSLOduration=2.207506681 podStartE2EDuration="6.883895985s" totalImagesPullingTime="3.608401271s" totalInitContainerRuntime="1.067988033s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:11:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:11:52.881141704 +0000 UTC m=+303.864112543" watchObservedRunningTime="2026-07-14 11:11:52.883895985 +0000 UTC m=+303.866866881" Jul 14 11:12:03.822438 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:03.822405 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7ff488f585-vrcv2" Jul 14 11:12:06.142543 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.142514 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz"] Jul 14 11:12:06.151718 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.151690 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.153320 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.153291 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz"] Jul 14 11:12:06.154938 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.154918 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d7gxs\"" Jul 14 11:12:06.155069 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.155041 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 11:12:06.155351 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.155326 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 11:12:06.155453 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.155433 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 11:12:06.296712 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.296686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/af10a7fc-cda8-4954-b038-4627c4c8b59c-manager-config\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.296712 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.296712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af10a7fc-cda8-4954-b038-4627c4c8b59c-cert\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.296867 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.296735 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhdk4\" (UniqueName: \"kubernetes.io/projected/af10a7fc-cda8-4954-b038-4627c4c8b59c-kube-api-access-zhdk4\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.296867 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.296823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/af10a7fc-cda8-4954-b038-4627c4c8b59c-metrics-cert\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.397593 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.397519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/af10a7fc-cda8-4954-b038-4627c4c8b59c-manager-config\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.397593 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.397554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af10a7fc-cda8-4954-b038-4627c4c8b59c-cert\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.397593 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.397585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhdk4\" (UniqueName: \"kubernetes.io/projected/af10a7fc-cda8-4954-b038-4627c4c8b59c-kube-api-access-zhdk4\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.397816 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.397645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/af10a7fc-cda8-4954-b038-4627c4c8b59c-metrics-cert\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.398155 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.398134 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/af10a7fc-cda8-4954-b038-4627c4c8b59c-manager-config\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.399984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.399963 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/af10a7fc-cda8-4954-b038-4627c4c8b59c-metrics-cert\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.400056 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.399997 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af10a7fc-cda8-4954-b038-4627c4c8b59c-cert\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.410026 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.410006 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhdk4\" (UniqueName: \"kubernetes.io/projected/af10a7fc-cda8-4954-b038-4627c4c8b59c-kube-api-access-zhdk4\") pod \"lws-controller-manager-7d84b86c7f-jrsrz\" (UID: \"af10a7fc-cda8-4954-b038-4627c4c8b59c\") " pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.462890 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.462865 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:06.614910 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.614885 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz"] Jul 14 11:12:06.617622 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:12:06.617595 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf10a7fc_cda8_4954_b038_4627c4c8b59c.slice/crio-1e73562adeff9b72fbfa8f5f0c138eb055e1cb4ab29a8a06c65d2f460456599e WatchSource:0}: Error finding container 1e73562adeff9b72fbfa8f5f0c138eb055e1cb4ab29a8a06c65d2f460456599e: Status 404 returned error can't find the container with id 1e73562adeff9b72fbfa8f5f0c138eb055e1cb4ab29a8a06c65d2f460456599e Jul 14 11:12:06.884045 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:06.884010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" event={"ID":"af10a7fc-cda8-4954-b038-4627c4c8b59c","Type":"ContainerStarted","Data":"1e73562adeff9b72fbfa8f5f0c138eb055e1cb4ab29a8a06c65d2f460456599e"} Jul 14 11:12:08.896187 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:08.896151 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" event={"ID":"af10a7fc-cda8-4954-b038-4627c4c8b59c","Type":"ContainerStarted","Data":"c7356e79244c516285c9cda6a2bea395a0a2bc566385d5a9ce35100f8f333fa5"} Jul 14 11:12:08.896568 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:08.896228 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:08.925478 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:08.925426 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" podStartSLOduration=1.556390662 podStartE2EDuration="2.92541092s" totalImagesPullingTime="1.369020258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:08.923899654 +0000 UTC m=+319.906870490" watchObservedRunningTime="2026-07-14 11:12:08.92541092 +0000 UTC m=+319.908381755" Jul 14 11:12:16.743827 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.743791 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr"] Jul 14 11:12:16.747067 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.747051 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.756579 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756557 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 11:12:16.756666 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756557 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 11:12:16.756666 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756559 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 11:12:16.756832 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756818 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 11:12:16.756929 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756907 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7k8pq\"" Jul 14 11:12:16.757045 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756921 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 11:12:16.757045 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.756908 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 11:12:16.772034 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.772005 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr"] Jul 14 11:12:16.788385 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.788514 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.788514 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788429 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.788514 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.788514 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788478 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.788643 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b2cfdab-75cd-49be-8d20-472434f8a0f4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.788643 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.788554 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj9m8\" (UniqueName: \"kubernetes.io/projected/7b2cfdab-75cd-49be-8d20-472434f8a0f4-kube-api-access-bj9m8\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889316 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889353 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b2cfdab-75cd-49be-8d20-472434f8a0f4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889543 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj9m8\" (UniqueName: \"kubernetes.io/projected/7b2cfdab-75cd-49be-8d20-472434f8a0f4-kube-api-access-bj9m8\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.889842 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.889615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.890247 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.890220 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.891784 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.891762 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.892099 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.892078 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b2cfdab-75cd-49be-8d20-472434f8a0f4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.892233 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.892216 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.892981 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.892963 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7b2cfdab-75cd-49be-8d20-472434f8a0f4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.903567 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.903545 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b2cfdab-75cd-49be-8d20-472434f8a0f4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:16.903567 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:16.903555 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj9m8\" (UniqueName: \"kubernetes.io/projected/7b2cfdab-75cd-49be-8d20-472434f8a0f4-kube-api-access-bj9m8\") pod \"istiod-openshift-gateway-94bb8fbfd-x7gsr\" (UID: \"7b2cfdab-75cd-49be-8d20-472434f8a0f4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:17.056335 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:17.056264 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:17.269851 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:17.269820 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr"] Jul 14 11:12:17.270405 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:12:17.270381 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2cfdab_75cd_49be_8d20_472434f8a0f4.slice/crio-f57d48ec3f52093378593e521033190afa08bb3e0a2fe86e6d67ad7fa92724e2 WatchSource:0}: Error finding container f57d48ec3f52093378593e521033190afa08bb3e0a2fe86e6d67ad7fa92724e2: Status 404 returned error can't find the container with id f57d48ec3f52093378593e521033190afa08bb3e0a2fe86e6d67ad7fa92724e2 Jul 14 11:12:17.937297 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:17.937261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" event={"ID":"7b2cfdab-75cd-49be-8d20-472434f8a0f4","Type":"ContainerStarted","Data":"f57d48ec3f52093378593e521033190afa08bb3e0a2fe86e6d67ad7fa92724e2"} Jul 14 11:12:19.739983 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:19.739944 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:12:19.740315 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:19.740016 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:12:19.903705 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:19.903678 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7d84b86c7f-jrsrz" Jul 14 11:12:19.950280 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:19.950241 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" event={"ID":"7b2cfdab-75cd-49be-8d20-472434f8a0f4","Type":"ContainerStarted","Data":"6056e6256eb2144fb763bc7319322c9a07b6cd50f23928da184dadf7d9ac5e25"} Jul 14 11:12:19.950426 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:19.950308 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:12:20.087013 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:20.086925 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" podStartSLOduration=1.619550985 podStartE2EDuration="4.086910532s" totalImagesPullingTime="2.467359547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:12:20.085671583 +0000 UTC m=+331.068642419" watchObservedRunningTime="2026-07-14 11:12:20.086910532 +0000 UTC m=+331.069881367" Jul 14 11:12:20.956290 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:12:20.956262 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x7gsr" Jul 14 11:13:25.034928 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.034889 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw"] Jul 14 11:13:25.038291 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.038274 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.040546 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.040528 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 11:13:25.040910 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.040898 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 11:13:25.040960 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.040931 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hr6wg\"" Jul 14 11:13:25.049018 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.048990 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw"] Jul 14 11:13:25.180677 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.180647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.180677 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.180681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2x75\" (UniqueName: \"kubernetes.io/projected/dc49c2c0-6196-45ed-ba57-7d26852538bf-kube-api-access-r2x75\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.180898 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.180703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.281995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.281954 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.281995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.281999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2x75\" (UniqueName: \"kubernetes.io/projected/dc49c2c0-6196-45ed-ba57-7d26852538bf-kube-api-access-r2x75\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.282268 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.282019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.282396 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.282376 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.282442 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.282402 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.290232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.290170 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2x75\" (UniqueName: \"kubernetes.io/projected/dc49c2c0-6196-45ed-ba57-7d26852538bf-kube-api-access-r2x75\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.347363 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.347325 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:25.484952 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.484927 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw"] Jul 14 11:13:25.488185 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:13:25.488157 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc49c2c0_6196_45ed_ba57_7d26852538bf.slice/crio-ea22188ec95c42b015c485aa51b17834eac769e1af67684d75ef7f9d82122577 WatchSource:0}: Error finding container ea22188ec95c42b015c485aa51b17834eac769e1af67684d75ef7f9d82122577: Status 404 returned error can't find the container with id ea22188ec95c42b015c485aa51b17834eac769e1af67684d75ef7f9d82122577 Jul 14 11:13:25.635259 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.635228 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf"] Jul 14 11:13:25.638797 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.638778 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.646780 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.646745 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf"] Jul 14 11:13:25.787140 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.787087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.787339 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.787208 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrtb\" (UniqueName: \"kubernetes.io/projected/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-kube-api-access-drrtb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.787339 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.787279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.888186 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.888143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drrtb\" (UniqueName: \"kubernetes.io/projected/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-kube-api-access-drrtb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.888186 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.888189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.888398 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.888275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.888641 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.888619 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.888677 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.888638 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.896083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.896047 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrtb\" (UniqueName: \"kubernetes.io/projected/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-kube-api-access-drrtb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:25.971868 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:25.971839 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:26.031962 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.031517 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v"] Jul 14 11:13:26.035633 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.035613 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.044822 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.044788 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v"] Jul 14 11:13:26.089818 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.089770 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfs4c\" (UniqueName: \"kubernetes.io/projected/8136bff3-e8b5-46f3-bd19-757cc93ba612-kube-api-access-wfs4c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.089995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.089861 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.089995 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.089929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.115553 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.115522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf"] Jul 14 11:13:26.117498 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:13:26.117466 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0d193f_1f62_4aa7_b8f4_4a11600bb3a3.slice/crio-f2b22bd973d9e80fb4ab91dc8e7bef6e35e847c9556050c0c63bfa1374de1cdc WatchSource:0}: Error finding container f2b22bd973d9e80fb4ab91dc8e7bef6e35e847c9556050c0c63bfa1374de1cdc: Status 404 returned error can't find the container with id f2b22bd973d9e80fb4ab91dc8e7bef6e35e847c9556050c0c63bfa1374de1cdc Jul 14 11:13:26.191175 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.191138 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfs4c\" (UniqueName: \"kubernetes.io/projected/8136bff3-e8b5-46f3-bd19-757cc93ba612-kube-api-access-wfs4c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.191370 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.191194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.191370 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.191233 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.191581 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.191565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.191692 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.191668 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.199266 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.199237 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfs4c\" (UniqueName: \"kubernetes.io/projected/8136bff3-e8b5-46f3-bd19-757cc93ba612-kube-api-access-wfs4c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.270984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.270947 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerID="7de39fe5d4886aa57945effd396b987ef7ab4d28f1897f49c9e798329a7a3390" exitCode=0 Jul 14 11:13:26.271176 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.271033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" event={"ID":"dc49c2c0-6196-45ed-ba57-7d26852538bf","Type":"ContainerDied","Data":"7de39fe5d4886aa57945effd396b987ef7ab4d28f1897f49c9e798329a7a3390"} Jul 14 11:13:26.271176 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.271090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" event={"ID":"dc49c2c0-6196-45ed-ba57-7d26852538bf","Type":"ContainerStarted","Data":"ea22188ec95c42b015c485aa51b17834eac769e1af67684d75ef7f9d82122577"} Jul 14 11:13:26.272666 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.272641 2581 generic.go:356] "Generic (PLEG): container finished" podID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerID="dcef8c8bc8a6dcb0b097b888974d634a6f6903288c5054a60b3c94ac09aa4925" exitCode=0 Jul 14 11:13:26.272753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.272681 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" event={"ID":"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3","Type":"ContainerDied","Data":"dcef8c8bc8a6dcb0b097b888974d634a6f6903288c5054a60b3c94ac09aa4925"} Jul 14 11:13:26.272753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.272714 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" event={"ID":"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3","Type":"ContainerStarted","Data":"f2b22bd973d9e80fb4ab91dc8e7bef6e35e847c9556050c0c63bfa1374de1cdc"} Jul 14 11:13:26.345999 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.345962 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:26.437487 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.437452 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5"] Jul 14 11:13:26.441943 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.441911 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.448927 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.448883 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5"] Jul 14 11:13:26.489739 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.489714 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v"] Jul 14 11:13:26.491562 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:13:26.491538 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8136bff3_e8b5_46f3_bd19_757cc93ba612.slice/crio-78ec92d381f90af6f1a1ffc3f27e91aa375c33d82919cd98ba981edf4d0471c0 WatchSource:0}: Error finding container 78ec92d381f90af6f1a1ffc3f27e91aa375c33d82919cd98ba981edf4d0471c0: Status 404 returned error can't find the container with id 78ec92d381f90af6f1a1ffc3f27e91aa375c33d82919cd98ba981edf4d0471c0 Jul 14 11:13:26.493735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.493708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.493829 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.493772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9lf\" (UniqueName: \"kubernetes.io/projected/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-kube-api-access-fh9lf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.493885 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.493873 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.595310 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.595281 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.595447 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.595329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9lf\" (UniqueName: \"kubernetes.io/projected/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-kube-api-access-fh9lf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.595447 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.595373 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.595696 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.595678 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.595760 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.595740 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.607191 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.607153 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9lf\" (UniqueName: \"kubernetes.io/projected/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-kube-api-access-fh9lf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.755261 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.755179 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:26.900913 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:26.900890 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5"] Jul 14 11:13:26.902389 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:13:26.902368 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312d31bd_8ad4_4d96_9499_1f1dd2bd3223.slice/crio-b5845f5c1234af97aab71d42802e1f55e14243cafbbce168ce320858f1d4b28c WatchSource:0}: Error finding container b5845f5c1234af97aab71d42802e1f55e14243cafbbce168ce320858f1d4b28c: Status 404 returned error can't find the container with id b5845f5c1234af97aab71d42802e1f55e14243cafbbce168ce320858f1d4b28c Jul 14 11:13:27.279784 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.279751 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerID="d6e7c4925af0a6af6b5a6e92d2fb561b0473e8f321ca5f5177a439d1460a5974" exitCode=0 Jul 14 11:13:27.280209 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.279820 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" event={"ID":"dc49c2c0-6196-45ed-ba57-7d26852538bf","Type":"ContainerDied","Data":"d6e7c4925af0a6af6b5a6e92d2fb561b0473e8f321ca5f5177a439d1460a5974"} Jul 14 11:13:27.281532 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.281411 2581 generic.go:356] "Generic (PLEG): container finished" podID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerID="a793a870edde4482ec6512a3fb4f9d607e47caff82f90610603d9666b85cdd91" exitCode=0 Jul 14 11:13:27.281532 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.281501 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" event={"ID":"312d31bd-8ad4-4d96-9499-1f1dd2bd3223","Type":"ContainerDied","Data":"a793a870edde4482ec6512a3fb4f9d607e47caff82f90610603d9666b85cdd91"} Jul 14 11:13:27.281655 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.281542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" event={"ID":"312d31bd-8ad4-4d96-9499-1f1dd2bd3223","Type":"ContainerStarted","Data":"b5845f5c1234af97aab71d42802e1f55e14243cafbbce168ce320858f1d4b28c"} Jul 14 11:13:27.283126 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.283088 2581 generic.go:356] "Generic (PLEG): container finished" podID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerID="5d4337ef9cdd84dc7d95d3f85f4fd28c1a6e5831c734613ad7c7729e0e1dc5ee" exitCode=0 Jul 14 11:13:27.283230 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.283164 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" event={"ID":"8136bff3-e8b5-46f3-bd19-757cc93ba612","Type":"ContainerDied","Data":"5d4337ef9cdd84dc7d95d3f85f4fd28c1a6e5831c734613ad7c7729e0e1dc5ee"} Jul 14 11:13:27.283230 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.283216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" event={"ID":"8136bff3-e8b5-46f3-bd19-757cc93ba612","Type":"ContainerStarted","Data":"78ec92d381f90af6f1a1ffc3f27e91aa375c33d82919cd98ba981edf4d0471c0"} Jul 14 11:13:27.285339 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.285319 2581 generic.go:356] "Generic (PLEG): container finished" podID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerID="bff220be5300061ae287d33f0a79b6b07b1e17d70acf583a767d522b343d7414" exitCode=0 Jul 14 11:13:27.285419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:27.285400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" event={"ID":"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3","Type":"ContainerDied","Data":"bff220be5300061ae287d33f0a79b6b07b1e17d70acf583a767d522b343d7414"} Jul 14 11:13:28.292809 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.292782 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerID="6c7005ef5c8772041450dbf8de1ed521b1867299389ce40b4f94d4c14e2d5cfb" exitCode=0 Jul 14 11:13:28.293211 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.292863 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" event={"ID":"dc49c2c0-6196-45ed-ba57-7d26852538bf","Type":"ContainerDied","Data":"6c7005ef5c8772041450dbf8de1ed521b1867299389ce40b4f94d4c14e2d5cfb"} Jul 14 11:13:28.294523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.294504 2581 generic.go:356] "Generic (PLEG): container finished" podID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerID="b211d8df8e294044381f35104818f30161533d8d83e39d0c67abf8327d752c4d" exitCode=0 Jul 14 11:13:28.294628 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.294573 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" event={"ID":"312d31bd-8ad4-4d96-9499-1f1dd2bd3223","Type":"ContainerDied","Data":"b211d8df8e294044381f35104818f30161533d8d83e39d0c67abf8327d752c4d"} Jul 14 11:13:28.296350 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.296330 2581 generic.go:356] "Generic (PLEG): container finished" podID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerID="cb98e41acc92da47d25543185ff2217d4580ba37e4388d24ca5f96e5d4237fd8" exitCode=0 Jul 14 11:13:28.296443 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.296404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" event={"ID":"8136bff3-e8b5-46f3-bd19-757cc93ba612","Type":"ContainerDied","Data":"cb98e41acc92da47d25543185ff2217d4580ba37e4388d24ca5f96e5d4237fd8"} Jul 14 11:13:28.298716 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.298696 2581 generic.go:356] "Generic (PLEG): container finished" podID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerID="c93056bfb205572d2dcf1f3d101957c2345cc0645d28808e1fd81cb442e65b4b" exitCode=0 Jul 14 11:13:28.298798 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:28.298727 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" event={"ID":"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3","Type":"ContainerDied","Data":"c93056bfb205572d2dcf1f3d101957c2345cc0645d28808e1fd81cb442e65b4b"} Jul 14 11:13:29.305911 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.305873 2581 generic.go:356] "Generic (PLEG): container finished" podID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerID="9164704cc7728db4962ace0070a6edf4ec5fa7271279fab3dca4bea82b3ec80e" exitCode=0 Jul 14 11:13:29.306385 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.305955 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" event={"ID":"8136bff3-e8b5-46f3-bd19-757cc93ba612","Type":"ContainerDied","Data":"9164704cc7728db4962ace0070a6edf4ec5fa7271279fab3dca4bea82b3ec80e"} Jul 14 11:13:29.308056 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.308036 2581 generic.go:356] "Generic (PLEG): container finished" podID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerID="07d0e527b8d89c37d86d2c9578ba3da424a6daf36a72866ecbdd120acc94ba9e" exitCode=0 Jul 14 11:13:29.308197 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.308136 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" event={"ID":"312d31bd-8ad4-4d96-9499-1f1dd2bd3223","Type":"ContainerDied","Data":"07d0e527b8d89c37d86d2c9578ba3da424a6daf36a72866ecbdd120acc94ba9e"} Jul 14 11:13:29.469062 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.469029 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:29.472020 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.472000 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:29.621722 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.621700 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-util\") pod \"dc49c2c0-6196-45ed-ba57-7d26852538bf\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " Jul 14 11:13:29.621855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.621739 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2x75\" (UniqueName: \"kubernetes.io/projected/dc49c2c0-6196-45ed-ba57-7d26852538bf-kube-api-access-r2x75\") pod \"dc49c2c0-6196-45ed-ba57-7d26852538bf\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " Jul 14 11:13:29.621855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.621764 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drrtb\" (UniqueName: \"kubernetes.io/projected/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-kube-api-access-drrtb\") pod \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " Jul 14 11:13:29.621855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.621802 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-bundle\") pod \"dc49c2c0-6196-45ed-ba57-7d26852538bf\" (UID: \"dc49c2c0-6196-45ed-ba57-7d26852538bf\") " Jul 14 11:13:29.621855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.621818 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-util\") pod \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " Jul 14 11:13:29.622039 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.621873 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-bundle\") pod \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\" (UID: \"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3\") " Jul 14 11:13:29.622538 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.622464 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-bundle" (OuterVolumeSpecName: "bundle") pod "df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" (UID: "df0d193f-1f62-4aa7-b8f4-4a11600bb3a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:29.622538 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.622475 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-bundle" (OuterVolumeSpecName: "bundle") pod "dc49c2c0-6196-45ed-ba57-7d26852538bf" (UID: "dc49c2c0-6196-45ed-ba57-7d26852538bf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:29.624361 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.624336 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-kube-api-access-drrtb" (OuterVolumeSpecName: "kube-api-access-drrtb") pod "df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" (UID: "df0d193f-1f62-4aa7-b8f4-4a11600bb3a3"). InnerVolumeSpecName "kube-api-access-drrtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:13:29.624361 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.624348 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc49c2c0-6196-45ed-ba57-7d26852538bf-kube-api-access-r2x75" (OuterVolumeSpecName: "kube-api-access-r2x75") pod "dc49c2c0-6196-45ed-ba57-7d26852538bf" (UID: "dc49c2c0-6196-45ed-ba57-7d26852538bf"). InnerVolumeSpecName "kube-api-access-r2x75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:13:29.627809 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.627774 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-util" (OuterVolumeSpecName: "util") pod "dc49c2c0-6196-45ed-ba57-7d26852538bf" (UID: "dc49c2c0-6196-45ed-ba57-7d26852538bf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:29.627874 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.627831 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-util" (OuterVolumeSpecName: "util") pod "df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" (UID: "df0d193f-1f62-4aa7-b8f4-4a11600bb3a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:29.723066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.723025 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:29.723066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.723058 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2x75\" (UniqueName: \"kubernetes.io/projected/dc49c2c0-6196-45ed-ba57-7d26852538bf-kube-api-access-r2x75\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:29.723066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.723069 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drrtb\" (UniqueName: \"kubernetes.io/projected/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-kube-api-access-drrtb\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:29.723066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.723078 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc49c2c0-6196-45ed-ba57-7d26852538bf-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:29.723362 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.723087 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:29.723362 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:29.723094 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df0d193f-1f62-4aa7-b8f4-4a11600bb3a3-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:30.314428 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.314390 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" Jul 14 11:13:30.314889 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.314395 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw" event={"ID":"dc49c2c0-6196-45ed-ba57-7d26852538bf","Type":"ContainerDied","Data":"ea22188ec95c42b015c485aa51b17834eac769e1af67684d75ef7f9d82122577"} Jul 14 11:13:30.314889 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.314537 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea22188ec95c42b015c485aa51b17834eac769e1af67684d75ef7f9d82122577" Jul 14 11:13:30.316318 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.316295 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" Jul 14 11:13:30.316459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.316322 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf" event={"ID":"df0d193f-1f62-4aa7-b8f4-4a11600bb3a3","Type":"ContainerDied","Data":"f2b22bd973d9e80fb4ab91dc8e7bef6e35e847c9556050c0c63bfa1374de1cdc"} Jul 14 11:13:30.316459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.316365 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2b22bd973d9e80fb4ab91dc8e7bef6e35e847c9556050c0c63bfa1374de1cdc" Jul 14 11:13:30.480894 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.480862 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:30.484660 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.484641 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:30.632972 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.632934 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-bundle\") pod \"8136bff3-e8b5-46f3-bd19-757cc93ba612\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " Jul 14 11:13:30.633184 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.632998 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-util\") pod \"8136bff3-e8b5-46f3-bd19-757cc93ba612\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " Jul 14 11:13:30.633184 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.633019 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-bundle\") pod \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " Jul 14 11:13:30.633184 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.633102 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfs4c\" (UniqueName: \"kubernetes.io/projected/8136bff3-e8b5-46f3-bd19-757cc93ba612-kube-api-access-wfs4c\") pod \"8136bff3-e8b5-46f3-bd19-757cc93ba612\" (UID: \"8136bff3-e8b5-46f3-bd19-757cc93ba612\") " Jul 14 11:13:30.633336 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.633185 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh9lf\" (UniqueName: \"kubernetes.io/projected/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-kube-api-access-fh9lf\") pod \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " Jul 14 11:13:30.633336 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.633214 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-util\") pod \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\" (UID: \"312d31bd-8ad4-4d96-9499-1f1dd2bd3223\") " Jul 14 11:13:30.633698 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.633666 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-bundle" (OuterVolumeSpecName: "bundle") pod "312d31bd-8ad4-4d96-9499-1f1dd2bd3223" (UID: "312d31bd-8ad4-4d96-9499-1f1dd2bd3223"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:30.633874 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.633829 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-bundle" (OuterVolumeSpecName: "bundle") pod "8136bff3-e8b5-46f3-bd19-757cc93ba612" (UID: "8136bff3-e8b5-46f3-bd19-757cc93ba612"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:30.635378 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.635352 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-kube-api-access-fh9lf" (OuterVolumeSpecName: "kube-api-access-fh9lf") pod "312d31bd-8ad4-4d96-9499-1f1dd2bd3223" (UID: "312d31bd-8ad4-4d96-9499-1f1dd2bd3223"). InnerVolumeSpecName "kube-api-access-fh9lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:13:30.635490 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.635398 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8136bff3-e8b5-46f3-bd19-757cc93ba612-kube-api-access-wfs4c" (OuterVolumeSpecName: "kube-api-access-wfs4c") pod "8136bff3-e8b5-46f3-bd19-757cc93ba612" (UID: "8136bff3-e8b5-46f3-bd19-757cc93ba612"). InnerVolumeSpecName "kube-api-access-wfs4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:13:30.638707 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.638672 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-util" (OuterVolumeSpecName: "util") pod "312d31bd-8ad4-4d96-9499-1f1dd2bd3223" (UID: "312d31bd-8ad4-4d96-9499-1f1dd2bd3223"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:30.641160 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.641136 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-util" (OuterVolumeSpecName: "util") pod "8136bff3-e8b5-46f3-bd19-757cc93ba612" (UID: "8136bff3-e8b5-46f3-bd19-757cc93ba612"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:13:30.733967 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.733930 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:30.733967 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.733961 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:30.733967 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.733971 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfs4c\" (UniqueName: \"kubernetes.io/projected/8136bff3-e8b5-46f3-bd19-757cc93ba612-kube-api-access-wfs4c\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:30.734250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.733985 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh9lf\" (UniqueName: \"kubernetes.io/projected/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-kube-api-access-fh9lf\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:30.734250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.733993 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/312d31bd-8ad4-4d96-9499-1f1dd2bd3223-util\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:30.734250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:30.734003 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8136bff3-e8b5-46f3-bd19-757cc93ba612-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:13:31.322564 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:31.322536 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" Jul 14 11:13:31.322974 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:31.322535 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5" event={"ID":"312d31bd-8ad4-4d96-9499-1f1dd2bd3223","Type":"ContainerDied","Data":"b5845f5c1234af97aab71d42802e1f55e14243cafbbce168ce320858f1d4b28c"} Jul 14 11:13:31.322974 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:31.322652 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5845f5c1234af97aab71d42802e1f55e14243cafbbce168ce320858f1d4b28c" Jul 14 11:13:31.324504 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:31.324483 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" event={"ID":"8136bff3-e8b5-46f3-bd19-757cc93ba612","Type":"ContainerDied","Data":"78ec92d381f90af6f1a1ffc3f27e91aa375c33d82919cd98ba981edf4d0471c0"} Jul 14 11:13:31.324600 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:31.324521 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78ec92d381f90af6f1a1ffc3f27e91aa375c33d82919cd98ba981edf4d0471c0" Jul 14 11:13:31.324600 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:31.324487 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v" Jul 14 11:13:49.076858 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:13:49.076827 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-98885cdcb-wrqmn"] Jul 14 11:14:14.097748 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.097643 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-98885cdcb-wrqmn" podUID="e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" containerName="console" containerID="cri-o://7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a" gracePeriod=15 Jul 14 11:14:14.347871 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.347811 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:14:14.417448 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417415 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-serving-cert\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.417634 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417456 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lld7b\" (UniqueName: \"kubernetes.io/projected/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-kube-api-access-lld7b\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.417634 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417477 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-config\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.417634 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417507 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-oauth-config\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.417634 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417555 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-trusted-ca-bundle\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.417634 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417577 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-oauth-serving-cert\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.417895 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417644 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-service-ca\") pod \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\" (UID: \"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4\") " Jul 14 11:14:14.418006 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417965 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-config" (OuterVolumeSpecName: "console-config") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:14:14.418006 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417977 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:14:14.418142 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.417995 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:14:14.418255 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.418236 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-service-ca" (OuterVolumeSpecName: "service-ca") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:14:14.419687 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.419661 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:14:14.419813 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.419687 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:14:14.419813 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.419670 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-kube-api-access-lld7b" (OuterVolumeSpecName: "kube-api-access-lld7b") pod "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" (UID: "e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4"). InnerVolumeSpecName "kube-api-access-lld7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:14:14.518620 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518582 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.518620 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518613 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lld7b\" (UniqueName: \"kubernetes.io/projected/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-kube-api-access-lld7b\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.518620 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518625 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.518863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518633 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-console-oauth-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.518863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518643 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-trusted-ca-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.518863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518651 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-oauth-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.518863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.518660 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:14:14.539191 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.539155 2581 generic.go:356] "Generic (PLEG): container finished" podID="e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" containerID="7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a" exitCode=0 Jul 14 11:14:14.539356 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.539204 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98885cdcb-wrqmn" event={"ID":"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4","Type":"ContainerDied","Data":"7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a"} Jul 14 11:14:14.539356 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.539250 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98885cdcb-wrqmn" Jul 14 11:14:14.539356 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.539260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98885cdcb-wrqmn" event={"ID":"e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4","Type":"ContainerDied","Data":"62e9d8f0250a83f9474cfc6f863d4a382229aa67917470621ee1ca06e854b41b"} Jul 14 11:14:14.539356 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.539291 2581 scope.go:117] "RemoveContainer" containerID="7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a" Jul 14 11:14:14.548201 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.548183 2581 scope.go:117] "RemoveContainer" containerID="7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a" Jul 14 11:14:14.548470 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:14:14.548451 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a\": container with ID starting with 7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a not found: ID does not exist" containerID="7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a" Jul 14 11:14:14.548509 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.548482 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a"} err="failed to get container status \"7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a\": rpc error: code = NotFound desc = could not find container \"7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a\": container with ID starting with 7e69eba80e1835c981c51d19747a69751fdbc27b43faca55e03d1b8bf585869a not found: ID does not exist" Jul 14 11:14:14.567167 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.567128 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-98885cdcb-wrqmn"] Jul 14 11:14:14.571806 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:14.571772 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-98885cdcb-wrqmn"] Jul 14 11:14:15.492578 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:14:15.492535 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" path="/var/lib/kubelet/pods/e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4/volumes" Jul 14 11:15:03.669259 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669223 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr"] Jul 14 11:15:03.669735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669677 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="pull" Jul 14 11:15:03.669735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669696 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="pull" Jul 14 11:15:03.669735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669712 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="extract" Jul 14 11:15:03.669735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669721 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="extract" Jul 14 11:15:03.669735 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669731 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="extract" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669739 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="extract" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669754 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" containerName="console" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669761 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" containerName="console" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669769 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669777 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669797 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="extract" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669805 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="extract" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669814 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669823 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669833 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="pull" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669840 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="pull" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669856 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="pull" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669864 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="pull" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669876 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669883 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669892 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="pull" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669899 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="pull" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669911 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="extract" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669920 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="extract" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669933 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="util" Jul 14 11:15:03.669993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.669941 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="util" Jul 14 11:15:03.670953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.670016 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8136bff3-e8b5-46f3-bd19-757cc93ba612" containerName="extract" Jul 14 11:15:03.670953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.670030 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0d193f-1f62-4aa7-b8f4-4a11600bb3a3" containerName="extract" Jul 14 11:15:03.670953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.670042 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="312d31bd-8ad4-4d96-9499-1f1dd2bd3223" containerName="extract" Jul 14 11:15:03.670953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.670053 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc49c2c0-6196-45ed-ba57-7d26852538bf" containerName="extract" Jul 14 11:15:03.670953 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.670060 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1a73c7d-17c0-4c3d-be0f-1efd45d9b6e4" containerName="console" Jul 14 11:15:03.674745 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.674722 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.677437 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.677409 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 11:15:03.677544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.677414 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 14 11:15:03.677937 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.677918 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 11:15:03.678020 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.677971 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5lv6m\"" Jul 14 11:15:03.681336 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.681313 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr"] Jul 14 11:15:03.683319 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.683244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-854fn\" (UniqueName: \"kubernetes.io/projected/11542375-3325-45f2-9c0e-fdfec9f2dc39-kube-api-access-854fn\") pod \"postgres-69fb4594cf-lkmbr\" (UID: \"11542375-3325-45f2-9c0e-fdfec9f2dc39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.683418 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.683364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/11542375-3325-45f2-9c0e-fdfec9f2dc39-data\") pod \"postgres-69fb4594cf-lkmbr\" (UID: \"11542375-3325-45f2-9c0e-fdfec9f2dc39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.784234 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.784202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/11542375-3325-45f2-9c0e-fdfec9f2dc39-data\") pod \"postgres-69fb4594cf-lkmbr\" (UID: \"11542375-3325-45f2-9c0e-fdfec9f2dc39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.784388 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.784250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-854fn\" (UniqueName: \"kubernetes.io/projected/11542375-3325-45f2-9c0e-fdfec9f2dc39-kube-api-access-854fn\") pod \"postgres-69fb4594cf-lkmbr\" (UID: \"11542375-3325-45f2-9c0e-fdfec9f2dc39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.784604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.784589 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/11542375-3325-45f2-9c0e-fdfec9f2dc39-data\") pod \"postgres-69fb4594cf-lkmbr\" (UID: \"11542375-3325-45f2-9c0e-fdfec9f2dc39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.793624 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.793595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-854fn\" (UniqueName: \"kubernetes.io/projected/11542375-3325-45f2-9c0e-fdfec9f2dc39-kube-api-access-854fn\") pod \"postgres-69fb4594cf-lkmbr\" (UID: \"11542375-3325-45f2-9c0e-fdfec9f2dc39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:03.987341 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:03.987267 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:04.128044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:04.128019 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr"] Jul 14 11:15:04.129935 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:15:04.129905 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11542375_3325_45f2_9c0e_fdfec9f2dc39.slice/crio-b2001b937fae08cdade6658f59241a795c7f131d12626ad2169f0457d7ffa1f7 WatchSource:0}: Error finding container b2001b937fae08cdade6658f59241a795c7f131d12626ad2169f0457d7ffa1f7: Status 404 returned error can't find the container with id b2001b937fae08cdade6658f59241a795c7f131d12626ad2169f0457d7ffa1f7 Jul 14 11:15:04.774601 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:04.774568 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" event={"ID":"11542375-3325-45f2-9c0e-fdfec9f2dc39","Type":"ContainerStarted","Data":"b2001b937fae08cdade6658f59241a795c7f131d12626ad2169f0457d7ffa1f7"} Jul 14 11:15:09.807399 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:09.807360 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" event={"ID":"11542375-3325-45f2-9c0e-fdfec9f2dc39","Type":"ContainerStarted","Data":"c44b63815561330ecc15fa65403dee53070728d46ef1ca6d79fa4745968e9308"} Jul 14 11:15:09.807850 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:09.807509 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:15:09.823251 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:09.823205 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" podStartSLOduration=1.69865569 podStartE2EDuration="6.823192976s" totalImagesPullingTime="5.124537286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:15:09.820957344 +0000 UTC m=+500.803928203" watchObservedRunningTime="2026-07-14 11:15:09.823192976 +0000 UTC m=+500.806163811" Jul 14 11:15:15.841052 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:15:15.841013 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lkmbr" Jul 14 11:16:00.621164 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.621067 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c786b4db6-fflpp"] Jul 14 11:16:00.623649 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.623630 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.626702 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.626685 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 14 11:16:00.627053 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.627039 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-shs8f\"" Jul 14 11:16:00.643522 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.643498 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c786b4db6-fflpp"] Jul 14 11:16:00.739466 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.739439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd94c02-7e09-4096-a998-651ae8c3aced-openshift-service-ca\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.739612 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.739477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94f72\" (UniqueName: \"kubernetes.io/projected/edd94c02-7e09-4096-a998-651ae8c3aced-kube-api-access-94f72\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.739612 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.739545 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd94c02-7e09-4096-a998-651ae8c3aced-maas-api-tls\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.840263 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.840235 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd94c02-7e09-4096-a998-651ae8c3aced-maas-api-tls\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.840425 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.840305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd94c02-7e09-4096-a998-651ae8c3aced-openshift-service-ca\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.840425 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.840329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94f72\" (UniqueName: \"kubernetes.io/projected/edd94c02-7e09-4096-a998-651ae8c3aced-kube-api-access-94f72\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.840949 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.840929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd94c02-7e09-4096-a998-651ae8c3aced-openshift-service-ca\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.842382 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.842365 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd94c02-7e09-4096-a998-651ae8c3aced-maas-api-tls\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.850279 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.850257 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94f72\" (UniqueName: \"kubernetes.io/projected/edd94c02-7e09-4096-a998-651ae8c3aced-kube-api-access-94f72\") pod \"maas-api-c786b4db6-fflpp\" (UID: \"edd94c02-7e09-4096-a998-651ae8c3aced\") " pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:00.932944 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:00.932916 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:01.091130 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:01.091082 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c786b4db6-fflpp"] Jul 14 11:16:01.094174 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:16:01.094144 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd94c02_7e09_4096_a998_651ae8c3aced.slice/crio-a02df47e72225821f0de62b2c5a5d8379980d8e41ad712f1a96360aa435fbe7a WatchSource:0}: Error finding container a02df47e72225821f0de62b2c5a5d8379980d8e41ad712f1a96360aa435fbe7a: Status 404 returned error can't find the container with id a02df47e72225821f0de62b2c5a5d8379980d8e41ad712f1a96360aa435fbe7a Jul 14 11:16:02.057269 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:02.057221 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" event={"ID":"edd94c02-7e09-4096-a998-651ae8c3aced","Type":"ContainerStarted","Data":"a02df47e72225821f0de62b2c5a5d8379980d8e41ad712f1a96360aa435fbe7a"} Jul 14 11:16:03.063512 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:03.063430 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" event={"ID":"edd94c02-7e09-4096-a998-651ae8c3aced","Type":"ContainerStarted","Data":"7501693e221386bc5e97cf271610c599830f3624779cebd5e44b3e54aa10a8e6"} Jul 14 11:16:03.063936 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:03.063534 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:03.080863 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:03.080816 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" podStartSLOduration=1.472687082 podStartE2EDuration="3.080802426s" totalImagesPullingTime="1.608115344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:16:03.07886355 +0000 UTC m=+554.061834397" watchObservedRunningTime="2026-07-14 11:16:03.080802426 +0000 UTC m=+554.063773262" Jul 14 11:16:09.075026 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:09.075001 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c786b4db6-fflpp" Jul 14 11:16:35.928605 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.928574 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9"] Jul 14 11:16:35.932020 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.932001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:35.934601 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.934576 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ln5z2\"" Jul 14 11:16:35.934715 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.934599 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 11:16:35.934715 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.934620 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 11:16:35.934715 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.934649 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 11:16:35.950572 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.950549 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9"] Jul 14 11:16:35.998535 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.998498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:35.998652 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.998545 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7ckz\" (UniqueName: \"kubernetes.io/projected/4cfff449-d3ff-433c-a9df-6d3855008c22-kube-api-access-h7ckz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:35.998652 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.998578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:35.998652 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.998616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfff449-d3ff-433c-a9df-6d3855008c22-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:35.998805 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.998666 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:35.998805 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:35.998710 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.099923 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.099889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.099923 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.099926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7ckz\" (UniqueName: \"kubernetes.io/projected/4cfff449-d3ff-433c-a9df-6d3855008c22-kube-api-access-h7ckz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100207 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.099949 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100207 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.099969 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfff449-d3ff-433c-a9df-6d3855008c22-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100207 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.100076 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100207 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.100168 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100411 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.100391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100468 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.100423 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.100538 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.100522 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.102345 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.102325 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cfff449-d3ff-433c-a9df-6d3855008c22-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.102614 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.102595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cfff449-d3ff-433c-a9df-6d3855008c22-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.108246 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.108225 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7ckz\" (UniqueName: \"kubernetes.io/projected/4cfff449-d3ff-433c-a9df-6d3855008c22-kube-api-access-h7ckz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pxfr9\" (UID: \"4cfff449-d3ff-433c-a9df-6d3855008c22\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.242598 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.242539 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:36.371568 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:36.371537 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9"] Jul 14 11:16:36.374697 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:16:36.374674 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfff449_d3ff_433c_a9df_6d3855008c22.slice/crio-0c2341f0cfb989647178fb343cc4aecc7cccd14d276317783987ec735876e305 WatchSource:0}: Error finding container 0c2341f0cfb989647178fb343cc4aecc7cccd14d276317783987ec735876e305: Status 404 returned error can't find the container with id 0c2341f0cfb989647178fb343cc4aecc7cccd14d276317783987ec735876e305 Jul 14 11:16:37.222643 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:37.222611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" event={"ID":"4cfff449-d3ff-433c-a9df-6d3855008c22","Type":"ContainerStarted","Data":"0c2341f0cfb989647178fb343cc4aecc7cccd14d276317783987ec735876e305"} Jul 14 11:16:38.229062 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:38.229022 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" event={"ID":"4cfff449-d3ff-433c-a9df-6d3855008c22","Type":"ContainerStarted","Data":"8fc28408e149ee7a9c0f58cbf1e0e1bdb12532137bc4cdc68758ca3f8300ae74"} Jul 14 11:16:38.245618 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:38.245574 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" podStartSLOduration=2.229911088 podStartE2EDuration="3.245559821s" totalImagesPullingTime="1.015648733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:16:38.244629949 +0000 UTC m=+589.227600795" watchObservedRunningTime="2026-07-14 11:16:38.245559821 +0000 UTC m=+589.228530655" Jul 14 11:16:46.243214 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:46.243178 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:46.243604 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:46.243244 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:46.255623 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:46.255603 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:46.279623 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:46.279596 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pxfr9" Jul 14 11:16:50.518395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:50.518297 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:16:50.518395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:50.518346 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:16:50.533543 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:50.521556 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:16:50.533543 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:16:50.521744 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:17:04.635750 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.635720 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf"] Jul 14 11:17:04.639343 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.639322 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.641835 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.641813 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 11:17:04.649004 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.648981 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf"] Jul 14 11:17:04.750132 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.750068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwbsn\" (UniqueName: \"kubernetes.io/projected/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-kube-api-access-vwbsn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.750247 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.750145 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.750247 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.750180 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.750247 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.750211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.750352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.750287 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.750352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.750312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851393 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851365 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwbsn\" (UniqueName: \"kubernetes.io/projected/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-kube-api-access-vwbsn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851480 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851852 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851830 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851919 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851852 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.851958 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.851922 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.853565 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.853538 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.853917 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.853898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.859997 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.859978 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwbsn\" (UniqueName: \"kubernetes.io/projected/2978df22-5d81-4a08-823d-f2f6cf5a0fa9-kube-api-access-vwbsn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf\" (UID: \"2978df22-5d81-4a08-823d-f2f6cf5a0fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:04.952158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:04.952066 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:05.082406 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:05.082383 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf"] Jul 14 11:17:05.083953 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:17:05.083925 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2978df22_5d81_4a08_823d_f2f6cf5a0fa9.slice/crio-cd7a5a81cc1783866918020ab6cbf8689f4c1209153ebc27ef32c8fb445fe777 WatchSource:0}: Error finding container cd7a5a81cc1783866918020ab6cbf8689f4c1209153ebc27ef32c8fb445fe777: Status 404 returned error can't find the container with id cd7a5a81cc1783866918020ab6cbf8689f4c1209153ebc27ef32c8fb445fe777 Jul 14 11:17:05.085411 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:05.085394 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:17:05.360050 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:05.360026 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" event={"ID":"2978df22-5d81-4a08-823d-f2f6cf5a0fa9","Type":"ContainerStarted","Data":"5ef134e0b9f49abf07aaeb0bb728e8175e28e7929f69ebf4e6a3c89ae9a6bcf6"} Jul 14 11:17:05.360176 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:05.360069 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" event={"ID":"2978df22-5d81-4a08-823d-f2f6cf5a0fa9","Type":"ContainerStarted","Data":"cd7a5a81cc1783866918020ab6cbf8689f4c1209153ebc27ef32c8fb445fe777"} Jul 14 11:17:05.377701 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:05.377658 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" podStartSLOduration=1.205768564 podStartE2EDuration="1.377643488s" totalImagesPullingTime="171.874924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:17:05.375937331 +0000 UTC m=+616.358908169" watchObservedRunningTime="2026-07-14 11:17:05.377643488 +0000 UTC m=+616.360614351" Jul 14 11:17:14.952392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:14.952297 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:14.952392 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:14.952375 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:14.964945 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:14.964923 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:15.417698 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:15.417672 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf" Jul 14 11:17:19.617249 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.617219 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j"] Jul 14 11:17:19.619817 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.619796 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.621845 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.621822 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 11:17:19.633581 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.633560 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j"] Jul 14 11:17:19.668170 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.668150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.668269 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.668180 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.668269 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.668206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9ts2\" (UniqueName: \"kubernetes.io/projected/754c0a11-67de-42aa-aaf8-1e68f21c0acc-kube-api-access-x9ts2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.668381 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.668299 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.668381 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.668351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/754c0a11-67de-42aa-aaf8-1e68f21c0acc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.668487 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.668425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769217 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769335 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769226 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769335 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9ts2\" (UniqueName: \"kubernetes.io/projected/754c0a11-67de-42aa-aaf8-1e68f21c0acc-kube-api-access-x9ts2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769335 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769335 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769313 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/754c0a11-67de-42aa-aaf8-1e68f21c0acc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769529 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769676 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769654 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769745 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769672 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.769745 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.769723 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.771499 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.771480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/754c0a11-67de-42aa-aaf8-1e68f21c0acc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.771768 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.771750 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/754c0a11-67de-42aa-aaf8-1e68f21c0acc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.777684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.777663 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9ts2\" (UniqueName: \"kubernetes.io/projected/754c0a11-67de-42aa-aaf8-1e68f21c0acc-kube-api-access-x9ts2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-f6r9j\" (UID: \"754c0a11-67de-42aa-aaf8-1e68f21c0acc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:19.933060 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:19.933033 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:20.065225 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:20.065199 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j"] Jul 14 11:17:20.067668 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:17:20.067640 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod754c0a11_67de_42aa_aaf8_1e68f21c0acc.slice/crio-c16f2d2407cf122b1cc85d8dd253870a15d0828dcd5fb64299bb907a5ddd10df WatchSource:0}: Error finding container c16f2d2407cf122b1cc85d8dd253870a15d0828dcd5fb64299bb907a5ddd10df: Status 404 returned error can't find the container with id c16f2d2407cf122b1cc85d8dd253870a15d0828dcd5fb64299bb907a5ddd10df Jul 14 11:17:20.430531 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:20.430493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" event={"ID":"754c0a11-67de-42aa-aaf8-1e68f21c0acc","Type":"ContainerStarted","Data":"5bfcf66f4731b14b63b98ffd7588a63e92c7918dec1ca52c7eb9c10b04a7a70b"} Jul 14 11:17:20.430690 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:20.430541 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" event={"ID":"754c0a11-67de-42aa-aaf8-1e68f21c0acc","Type":"ContainerStarted","Data":"c16f2d2407cf122b1cc85d8dd253870a15d0828dcd5fb64299bb907a5ddd10df"} Jul 14 11:17:20.448573 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:20.448522 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" podStartSLOduration=1.269330861 podStartE2EDuration="1.448501836s" totalImagesPullingTime="179.170975ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:17:20.447700851 +0000 UTC m=+631.430671687" watchObservedRunningTime="2026-07-14 11:17:20.448501836 +0000 UTC m=+631.431472672" Jul 14 11:17:29.934167 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:29.934098 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:29.934687 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:29.934207 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:29.946827 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:29.946796 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:17:30.489664 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:17:30.489636 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-f6r9j" Jul 14 11:21:50.554544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:21:50.554449 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:21:50.554544 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:21:50.554453 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:21:50.560441 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:21:50.557612 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:21:50.560441 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:21:50.557673 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:26:50.586424 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:26:50.586328 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:26:50.590372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:26:50.587334 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:26:50.590372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:26:50.589310 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:26:50.590372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:26:50.590366 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:31:50.618044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:31:50.617931 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:31:50.621688 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:31:50.620063 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:31:50.621688 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:31:50.621083 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:31:50.623210 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:31:50.623193 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:32:17.621889 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.621854 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6"] Jul 14 11:32:17.625188 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.625167 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.628543 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.628525 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-02a04895-serving-cert\"" Jul 14 11:32:17.649984 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.649960 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6"] Jul 14 11:32:17.668011 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.667988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lds4\" (UniqueName: \"kubernetes.io/projected/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-kube-api-access-9lds4\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.668142 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.668020 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-openshift-service-ca\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.668142 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.668045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-maas-api-tls\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.769473 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.769439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lds4\" (UniqueName: \"kubernetes.io/projected/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-kube-api-access-9lds4\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.769651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.769478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-openshift-service-ca\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.769651 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.769514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-maas-api-tls\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.770158 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.770101 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-openshift-service-ca\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.772013 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.771987 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-maas-api-tls\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.789150 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.789108 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lds4\" (UniqueName: \"kubernetes.io/projected/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-kube-api-access-9lds4\") pod \"maas-api-e2e-ait-02a04895-56dff7c58d-996f6\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:17.934969 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:17.934889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:18.072947 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:18.072921 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6"] Jul 14 11:32:18.075186 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:32:18.075148 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc269f0dd_0c2b_4ec6_aa44_716aa893fd8e.slice/crio-c031072589e8365d37b891b51876bc6c7f672e412accf34f15ea8761ba13be16 WatchSource:0}: Error finding container c031072589e8365d37b891b51876bc6c7f672e412accf34f15ea8761ba13be16: Status 404 returned error can't find the container with id c031072589e8365d37b891b51876bc6c7f672e412accf34f15ea8761ba13be16 Jul 14 11:32:18.076493 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:18.076475 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:32:18.838400 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:18.838317 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" event={"ID":"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e","Type":"ContainerStarted","Data":"63e30a9f06eb3a3491cdda06607add9a6a92df1ef597144f799fce7397a04f43"} Jul 14 11:32:18.838400 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:18.838369 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" event={"ID":"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e","Type":"ContainerStarted","Data":"c031072589e8365d37b891b51876bc6c7f672e412accf34f15ea8761ba13be16"} Jul 14 11:32:18.838865 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:18.838417 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:32:18.874600 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:18.874550 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" podStartSLOduration=1.402606788 podStartE2EDuration="1.874534551s" totalImagesPullingTime="471.927763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:32:18.87118192 +0000 UTC m=+1529.854152756" watchObservedRunningTime="2026-07-14 11:32:18.874534551 +0000 UTC m=+1529.857505385" Jul 14 11:32:24.848527 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:32:24.848500 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:33:32.963640 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:32.963594 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6"] Jul 14 11:33:32.964095 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:32.963844 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" podUID="c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" containerName="maas-api" containerID="cri-o://63e30a9f06eb3a3491cdda06607add9a6a92df1ef597144f799fce7397a04f43" gracePeriod=30 Jul 14 11:33:33.201568 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.201537 2581 generic.go:356] "Generic (PLEG): container finished" podID="c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" containerID="63e30a9f06eb3a3491cdda06607add9a6a92df1ef597144f799fce7397a04f43" exitCode=0 Jul 14 11:33:33.201699 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.201603 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" event={"ID":"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e","Type":"ContainerDied","Data":"63e30a9f06eb3a3491cdda06607add9a6a92df1ef597144f799fce7397a04f43"} Jul 14 11:33:33.219799 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.219744 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:33:33.322226 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.322190 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-maas-api-tls\") pod \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " Jul 14 11:33:33.322226 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.322232 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lds4\" (UniqueName: \"kubernetes.io/projected/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-kube-api-access-9lds4\") pod \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " Jul 14 11:33:33.322489 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.322268 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-openshift-service-ca\") pod \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\" (UID: \"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e\") " Jul 14 11:33:33.322665 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.322641 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" (UID: "c269f0dd-0c2b-4ec6-aa44-716aa893fd8e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:33:33.324418 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.324392 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" (UID: "c269f0dd-0c2b-4ec6-aa44-716aa893fd8e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:33:33.324501 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.324452 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-kube-api-access-9lds4" (OuterVolumeSpecName: "kube-api-access-9lds4") pod "c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" (UID: "c269f0dd-0c2b-4ec6-aa44-716aa893fd8e"). InnerVolumeSpecName "kube-api-access-9lds4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:33:33.423702 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.423672 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:33:33.423702 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.423695 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lds4\" (UniqueName: \"kubernetes.io/projected/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-kube-api-access-9lds4\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:33:33.423702 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:33.423706 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:33:34.208071 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:34.208045 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" Jul 14 11:33:34.208071 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:34.208066 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6" event={"ID":"c269f0dd-0c2b-4ec6-aa44-716aa893fd8e","Type":"ContainerDied","Data":"c031072589e8365d37b891b51876bc6c7f672e412accf34f15ea8761ba13be16"} Jul 14 11:33:34.208568 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:34.208152 2581 scope.go:117] "RemoveContainer" containerID="63e30a9f06eb3a3491cdda06607add9a6a92df1ef597144f799fce7397a04f43" Jul 14 11:33:34.233579 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:34.233548 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6"] Jul 14 11:33:34.240352 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:34.240327 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-02a04895-56dff7c58d-996f6"] Jul 14 11:33:35.498004 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:35.497970 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" path="/var/lib/kubelet/pods/c269f0dd-0c2b-4ec6-aa44-716aa893fd8e/volumes" Jul 14 11:33:49.691980 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.691946 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd"] Jul 14 11:33:49.692378 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.692364 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" containerName="maas-api" Jul 14 11:33:49.692416 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.692381 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" containerName="maas-api" Jul 14 11:33:49.692460 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.692450 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c269f0dd-0c2b-4ec6-aa44-716aa893fd8e" containerName="maas-api" Jul 14 11:33:49.697010 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.696988 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.699669 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.699650 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c50a5e30-serving-cert\"" Jul 14 11:33:49.708990 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.708965 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd"] Jul 14 11:33:49.772827 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.772789 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfhmp\" (UniqueName: \"kubernetes.io/projected/628486b6-48f4-451e-80fc-10fa33985436-kube-api-access-jfhmp\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.772827 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.772829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.773061 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.772917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/628486b6-48f4-451e-80fc-10fa33985436-openshift-service-ca\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.873792 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.873763 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/628486b6-48f4-451e-80fc-10fa33985436-openshift-service-ca\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.873942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.873836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhmp\" (UniqueName: \"kubernetes.io/projected/628486b6-48f4-451e-80fc-10fa33985436-kube-api-access-jfhmp\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.873942 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.873854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.874031 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:33:49.874019 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-serving-cert: secret "maas-api-e2e-migrate-c50a5e30-serving-cert" not found Jul 14 11:33:49.874087 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:33:49.874078 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls podName:628486b6-48f4-451e-80fc-10fa33985436 nodeName:}" failed. No retries permitted until 2026-07-14 11:33:50.374063282 +0000 UTC m=+1621.357034095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls") pod "maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" (UID: "628486b6-48f4-451e-80fc-10fa33985436") : secret "maas-api-e2e-migrate-c50a5e30-serving-cert" not found Jul 14 11:33:49.874542 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.874525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/628486b6-48f4-451e-80fc-10fa33985436-openshift-service-ca\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:49.882454 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:49.882433 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhmp\" (UniqueName: \"kubernetes.io/projected/628486b6-48f4-451e-80fc-10fa33985436-kube-api-access-jfhmp\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:50.378752 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:50.378718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:50.381279 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:50.381259 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls\") pod \"maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:50.608044 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:50.608014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:50.744148 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:50.744103 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd"] Jul 14 11:33:50.746170 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:33:50.746142 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628486b6_48f4_451e_80fc_10fa33985436.slice/crio-9232ee414a4f421abfe365003380b53c88ed73ca5ad76cadab65cbc84f1b84d5 WatchSource:0}: Error finding container 9232ee414a4f421abfe365003380b53c88ed73ca5ad76cadab65cbc84f1b84d5: Status 404 returned error can't find the container with id 9232ee414a4f421abfe365003380b53c88ed73ca5ad76cadab65cbc84f1b84d5 Jul 14 11:33:51.295209 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:51.295180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" event={"ID":"628486b6-48f4-451e-80fc-10fa33985436","Type":"ContainerStarted","Data":"fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a"} Jul 14 11:33:51.295209 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:51.295225 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" event={"ID":"628486b6-48f4-451e-80fc-10fa33985436","Type":"ContainerStarted","Data":"9232ee414a4f421abfe365003380b53c88ed73ca5ad76cadab65cbc84f1b84d5"} Jul 14 11:33:51.295456 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:51.295353 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:33:51.318725 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:51.318672 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" podStartSLOduration=1.90794127 podStartE2EDuration="2.318658239s" totalImagesPullingTime="410.716969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:33:51.315923065 +0000 UTC m=+1622.298893899" watchObservedRunningTime="2026-07-14 11:33:51.318658239 +0000 UTC m=+1622.301629072" Jul 14 11:33:57.307358 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:33:57.307327 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:34:54.696501 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:54.696465 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd"] Jul 14 11:34:54.697029 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:54.696683 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" podUID="628486b6-48f4-451e-80fc-10fa33985436" containerName="maas-api" containerID="cri-o://fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a" gracePeriod=30 Jul 14 11:34:54.940523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:54.940502 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:34:55.018821 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.018755 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfhmp\" (UniqueName: \"kubernetes.io/projected/628486b6-48f4-451e-80fc-10fa33985436-kube-api-access-jfhmp\") pod \"628486b6-48f4-451e-80fc-10fa33985436\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " Jul 14 11:34:55.018821 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.018796 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls\") pod \"628486b6-48f4-451e-80fc-10fa33985436\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " Jul 14 11:34:55.019005 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.018839 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/628486b6-48f4-451e-80fc-10fa33985436-openshift-service-ca\") pod \"628486b6-48f4-451e-80fc-10fa33985436\" (UID: \"628486b6-48f4-451e-80fc-10fa33985436\") " Jul 14 11:34:55.019282 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.019252 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628486b6-48f4-451e-80fc-10fa33985436-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "628486b6-48f4-451e-80fc-10fa33985436" (UID: "628486b6-48f4-451e-80fc-10fa33985436"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:34:55.020789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.020761 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628486b6-48f4-451e-80fc-10fa33985436-kube-api-access-jfhmp" (OuterVolumeSpecName: "kube-api-access-jfhmp") pod "628486b6-48f4-451e-80fc-10fa33985436" (UID: "628486b6-48f4-451e-80fc-10fa33985436"). InnerVolumeSpecName "kube-api-access-jfhmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:34:55.020884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.020769 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "628486b6-48f4-451e-80fc-10fa33985436" (UID: "628486b6-48f4-451e-80fc-10fa33985436"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:34:55.120400 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.120367 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfhmp\" (UniqueName: \"kubernetes.io/projected/628486b6-48f4-451e-80fc-10fa33985436-kube-api-access-jfhmp\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:34:55.120400 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.120398 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/628486b6-48f4-451e-80fc-10fa33985436-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:34:55.120400 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.120407 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/628486b6-48f4-451e-80fc-10fa33985436-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:34:55.621386 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.621355 2581 generic.go:356] "Generic (PLEG): container finished" podID="628486b6-48f4-451e-80fc-10fa33985436" containerID="fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a" exitCode=0 Jul 14 11:34:55.621516 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.621409 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" Jul 14 11:34:55.621516 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.621418 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" event={"ID":"628486b6-48f4-451e-80fc-10fa33985436","Type":"ContainerDied","Data":"fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a"} Jul 14 11:34:55.621516 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.621474 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd" event={"ID":"628486b6-48f4-451e-80fc-10fa33985436","Type":"ContainerDied","Data":"9232ee414a4f421abfe365003380b53c88ed73ca5ad76cadab65cbc84f1b84d5"} Jul 14 11:34:55.621516 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.621499 2581 scope.go:117] "RemoveContainer" containerID="fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a" Jul 14 11:34:55.631524 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.631501 2581 scope.go:117] "RemoveContainer" containerID="fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a" Jul 14 11:34:55.631871 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:34:55.631854 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a\": container with ID starting with fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a not found: ID does not exist" containerID="fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a" Jul 14 11:34:55.631927 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.631880 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a"} err="failed to get container status \"fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a\": rpc error: code = NotFound desc = could not find container \"fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a\": container with ID starting with fac1a5343b89ce384b4b8a81192b11f21822a695b5c67d1e3fe468340c7b089a not found: ID does not exist" Jul 14 11:34:55.637563 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.637543 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd"] Jul 14 11:34:55.643465 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:55.643447 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c50a5e30-7c9d4d4b46-dngxd"] Jul 14 11:34:57.492641 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:34:57.492608 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628486b6-48f4-451e-80fc-10fa33985436" path="/var/lib/kubelet/pods/628486b6-48f4-451e-80fc-10fa33985436/volumes" Jul 14 11:35:02.089894 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.089854 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8"] Jul 14 11:35:02.090876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.090846 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628486b6-48f4-451e-80fc-10fa33985436" containerName="maas-api" Jul 14 11:35:02.090876 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.090879 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="628486b6-48f4-451e-80fc-10fa33985436" containerName="maas-api" Jul 14 11:35:02.094145 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.092064 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="628486b6-48f4-451e-80fc-10fa33985436" containerName="maas-api" Jul 14 11:35:02.098677 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.098656 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.101063 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.101043 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-57573546-serving-cert\"" Jul 14 11:35:02.116564 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.116541 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8"] Jul 14 11:35:02.173207 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.173179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/313f2f17-b358-419d-a098-022e97841804-openshift-service-ca\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.173351 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.173280 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxxvm\" (UniqueName: \"kubernetes.io/projected/313f2f17-b358-419d-a098-022e97841804-kube-api-access-nxxvm\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.173351 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.173304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/313f2f17-b358-419d-a098-022e97841804-maas-api-tls\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.274325 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.274299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxxvm\" (UniqueName: \"kubernetes.io/projected/313f2f17-b358-419d-a098-022e97841804-kube-api-access-nxxvm\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.274525 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.274335 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/313f2f17-b358-419d-a098-022e97841804-maas-api-tls\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.274525 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.274397 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/313f2f17-b358-419d-a098-022e97841804-openshift-service-ca\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.275059 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.275041 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/313f2f17-b358-419d-a098-022e97841804-openshift-service-ca\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.276783 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.276760 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/313f2f17-b358-419d-a098-022e97841804-maas-api-tls\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.282901 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.282877 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxxvm\" (UniqueName: \"kubernetes.io/projected/313f2f17-b358-419d-a098-022e97841804-kube-api-access-nxxvm\") pod \"maas-api-e2e-ait-57573546-66d87f69c9-5kcf8\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.408692 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.408664 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:02.750321 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:02.750297 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8"] Jul 14 11:35:02.752724 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:35:02.752695 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313f2f17_b358_419d_a098_022e97841804.slice/crio-b7a4535965a3bce53d2ef46547676a8d35d6c027936cadf265f8bba3fe096923 WatchSource:0}: Error finding container b7a4535965a3bce53d2ef46547676a8d35d6c027936cadf265f8bba3fe096923: Status 404 returned error can't find the container with id b7a4535965a3bce53d2ef46547676a8d35d6c027936cadf265f8bba3fe096923 Jul 14 11:35:03.664526 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:03.664490 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" event={"ID":"313f2f17-b358-419d-a098-022e97841804","Type":"ContainerStarted","Data":"2fad036a3fa1423b08154d327af74533edac5e5ae1c57116824884527e916216"} Jul 14 11:35:03.664885 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:03.664552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" event={"ID":"313f2f17-b358-419d-a098-022e97841804","Type":"ContainerStarted","Data":"b7a4535965a3bce53d2ef46547676a8d35d6c027936cadf265f8bba3fe096923"} Jul 14 11:35:03.664885 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:03.664585 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:03.689930 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:03.689882 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" podStartSLOduration=1.284494314 podStartE2EDuration="1.689862903s" totalImagesPullingTime="405.368589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:35:03.68831398 +0000 UTC m=+1694.671284824" watchObservedRunningTime="2026-07-14 11:35:03.689862903 +0000 UTC m=+1694.672833742" Jul 14 11:35:09.674956 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:09.674926 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:17.572840 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.572808 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8"] Jul 14 11:35:17.573332 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.573047 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" podUID="313f2f17-b358-419d-a098-022e97841804" containerName="maas-api" containerID="cri-o://2fad036a3fa1423b08154d327af74533edac5e5ae1c57116824884527e916216" gracePeriod=30 Jul 14 11:35:17.735718 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.735690 2581 generic.go:356] "Generic (PLEG): container finished" podID="313f2f17-b358-419d-a098-022e97841804" containerID="2fad036a3fa1423b08154d327af74533edac5e5ae1c57116824884527e916216" exitCode=0 Jul 14 11:35:17.735906 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.735751 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" event={"ID":"313f2f17-b358-419d-a098-022e97841804","Type":"ContainerDied","Data":"2fad036a3fa1423b08154d327af74533edac5e5ae1c57116824884527e916216"} Jul 14 11:35:17.818788 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.818757 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:17.901079 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.901053 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/313f2f17-b358-419d-a098-022e97841804-openshift-service-ca\") pod \"313f2f17-b358-419d-a098-022e97841804\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " Jul 14 11:35:17.901238 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.901092 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxxvm\" (UniqueName: \"kubernetes.io/projected/313f2f17-b358-419d-a098-022e97841804-kube-api-access-nxxvm\") pod \"313f2f17-b358-419d-a098-022e97841804\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " Jul 14 11:35:17.901238 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.901108 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/313f2f17-b358-419d-a098-022e97841804-maas-api-tls\") pod \"313f2f17-b358-419d-a098-022e97841804\" (UID: \"313f2f17-b358-419d-a098-022e97841804\") " Jul 14 11:35:17.901478 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.901456 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/313f2f17-b358-419d-a098-022e97841804-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "313f2f17-b358-419d-a098-022e97841804" (UID: "313f2f17-b358-419d-a098-022e97841804"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:35:17.903151 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.903128 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/313f2f17-b358-419d-a098-022e97841804-kube-api-access-nxxvm" (OuterVolumeSpecName: "kube-api-access-nxxvm") pod "313f2f17-b358-419d-a098-022e97841804" (UID: "313f2f17-b358-419d-a098-022e97841804"). InnerVolumeSpecName "kube-api-access-nxxvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:35:17.903232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:17.903131 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/313f2f17-b358-419d-a098-022e97841804-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "313f2f17-b358-419d-a098-022e97841804" (UID: "313f2f17-b358-419d-a098-022e97841804"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:35:18.001921 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.001898 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/313f2f17-b358-419d-a098-022e97841804-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:35:18.001921 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.001920 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxxvm\" (UniqueName: \"kubernetes.io/projected/313f2f17-b358-419d-a098-022e97841804-kube-api-access-nxxvm\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:35:18.002054 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.001930 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/313f2f17-b358-419d-a098-022e97841804-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:35:18.748408 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.748369 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" event={"ID":"313f2f17-b358-419d-a098-022e97841804","Type":"ContainerDied","Data":"b7a4535965a3bce53d2ef46547676a8d35d6c027936cadf265f8bba3fe096923"} Jul 14 11:35:18.748822 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.748413 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8" Jul 14 11:35:18.748822 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.748467 2581 scope.go:117] "RemoveContainer" containerID="2fad036a3fa1423b08154d327af74533edac5e5ae1c57116824884527e916216" Jul 14 11:35:18.772643 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.772622 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8"] Jul 14 11:35:18.774545 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:18.774527 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57573546-66d87f69c9-5kcf8"] Jul 14 11:35:19.491342 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:19.491312 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="313f2f17-b358-419d-a098-022e97841804" path="/var/lib/kubelet/pods/313f2f17-b358-419d-a098-022e97841804/volumes" Jul 14 11:35:34.774274 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.774237 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j"] Jul 14 11:35:34.774789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.774765 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="313f2f17-b358-419d-a098-022e97841804" containerName="maas-api" Jul 14 11:35:34.774789 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.774790 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="313f2f17-b358-419d-a098-022e97841804" containerName="maas-api" Jul 14 11:35:34.774940 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.774920 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="313f2f17-b358-419d-a098-022e97841804" containerName="maas-api" Jul 14 11:35:34.778002 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.777982 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:34.780064 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.780043 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1cf7269b-serving-cert\"" Jul 14 11:35:34.787514 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.787492 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j"] Jul 14 11:35:34.945861 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.945807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:34.946066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.945893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm458\" (UniqueName: \"kubernetes.io/projected/d31180f4-2ac3-4df1-81e8-1a01edf96f73-kube-api-access-cm458\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:34.946066 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:34.945961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31180f4-2ac3-4df1-81e8-1a01edf96f73-openshift-service-ca\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.047029 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.046919 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.047029 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.046986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm458\" (UniqueName: \"kubernetes.io/projected/d31180f4-2ac3-4df1-81e8-1a01edf96f73-kube-api-access-cm458\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.047029 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.047038 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31180f4-2ac3-4df1-81e8-1a01edf96f73-openshift-service-ca\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.047336 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:35:35.047092 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-serving-cert: secret "maas-api-e2e-derive-1cf7269b-serving-cert" not found Jul 14 11:35:35.047336 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:35:35.047210 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls podName:d31180f4-2ac3-4df1-81e8-1a01edf96f73 nodeName:}" failed. No retries permitted until 2026-07-14 11:35:35.54718809 +0000 UTC m=+1726.530158918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls") pod "maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" (UID: "d31180f4-2ac3-4df1-81e8-1a01edf96f73") : secret "maas-api-e2e-derive-1cf7269b-serving-cert" not found Jul 14 11:35:35.047647 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.047630 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31180f4-2ac3-4df1-81e8-1a01edf96f73-openshift-service-ca\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.058360 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.058330 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm458\" (UniqueName: \"kubernetes.io/projected/d31180f4-2ac3-4df1-81e8-1a01edf96f73-kube-api-access-cm458\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.551482 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.551441 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.553965 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.553938 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls\") pod \"maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.689874 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.689841 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:35.835198 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:35.835174 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j"] Jul 14 11:35:35.837165 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:35:35.837109 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31180f4_2ac3_4df1_81e8_1a01edf96f73.slice/crio-f2a0557960b241ed5d4d613a22baec376783626d71ff34df1156963c8d407136 WatchSource:0}: Error finding container f2a0557960b241ed5d4d613a22baec376783626d71ff34df1156963c8d407136: Status 404 returned error can't find the container with id f2a0557960b241ed5d4d613a22baec376783626d71ff34df1156963c8d407136 Jul 14 11:35:36.838783 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:36.838748 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" event={"ID":"d31180f4-2ac3-4df1-81e8-1a01edf96f73","Type":"ContainerStarted","Data":"fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42"} Jul 14 11:35:36.839153 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:36.838801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" event={"ID":"d31180f4-2ac3-4df1-81e8-1a01edf96f73","Type":"ContainerStarted","Data":"f2a0557960b241ed5d4d613a22baec376783626d71ff34df1156963c8d407136"} Jul 14 11:35:36.839153 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:36.838827 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:35:36.857381 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:36.857329 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" podStartSLOduration=2.4940496850000002 podStartE2EDuration="2.857310305s" totalImagesPullingTime="363.26062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:35:36.856985939 +0000 UTC m=+1727.839956773" watchObservedRunningTime="2026-07-14 11:35:36.857310305 +0000 UTC m=+1727.840281141" Jul 14 11:35:42.850372 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:35:42.850342 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:36:39.607043 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.607003 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j"] Jul 14 11:36:39.607557 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.607393 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" podUID="d31180f4-2ac3-4df1-81e8-1a01edf96f73" containerName="maas-api" containerID="cri-o://fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42" gracePeriod=30 Jul 14 11:36:39.850909 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.850883 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:36:39.981955 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.981917 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls\") pod \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " Jul 14 11:36:39.982182 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.981979 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm458\" (UniqueName: \"kubernetes.io/projected/d31180f4-2ac3-4df1-81e8-1a01edf96f73-kube-api-access-cm458\") pod \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " Jul 14 11:36:39.982182 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.982141 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31180f4-2ac3-4df1-81e8-1a01edf96f73-openshift-service-ca\") pod \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\" (UID: \"d31180f4-2ac3-4df1-81e8-1a01edf96f73\") " Jul 14 11:36:39.982536 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.982512 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31180f4-2ac3-4df1-81e8-1a01edf96f73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d31180f4-2ac3-4df1-81e8-1a01edf96f73" (UID: "d31180f4-2ac3-4df1-81e8-1a01edf96f73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:36:39.983976 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.983957 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d31180f4-2ac3-4df1-81e8-1a01edf96f73" (UID: "d31180f4-2ac3-4df1-81e8-1a01edf96f73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:36:39.984296 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:39.984276 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31180f4-2ac3-4df1-81e8-1a01edf96f73-kube-api-access-cm458" (OuterVolumeSpecName: "kube-api-access-cm458") pod "d31180f4-2ac3-4df1-81e8-1a01edf96f73" (UID: "d31180f4-2ac3-4df1-81e8-1a01edf96f73"). InnerVolumeSpecName "kube-api-access-cm458". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:36:40.083596 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.083562 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31180f4-2ac3-4df1-81e8-1a01edf96f73-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:36:40.083596 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.083594 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm458\" (UniqueName: \"kubernetes.io/projected/d31180f4-2ac3-4df1-81e8-1a01edf96f73-kube-api-access-cm458\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:36:40.083596 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.083606 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31180f4-2ac3-4df1-81e8-1a01edf96f73-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:36:40.150132 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.150085 2581 generic.go:356] "Generic (PLEG): container finished" podID="d31180f4-2ac3-4df1-81e8-1a01edf96f73" containerID="fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42" exitCode=0 Jul 14 11:36:40.150321 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.150138 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" Jul 14 11:36:40.150321 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.150171 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" event={"ID":"d31180f4-2ac3-4df1-81e8-1a01edf96f73","Type":"ContainerDied","Data":"fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42"} Jul 14 11:36:40.150321 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.150225 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j" event={"ID":"d31180f4-2ac3-4df1-81e8-1a01edf96f73","Type":"ContainerDied","Data":"f2a0557960b241ed5d4d613a22baec376783626d71ff34df1156963c8d407136"} Jul 14 11:36:40.150321 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.150247 2581 scope.go:117] "RemoveContainer" containerID="fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42" Jul 14 11:36:40.159639 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.159622 2581 scope.go:117] "RemoveContainer" containerID="fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42" Jul 14 11:36:40.159908 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:36:40.159891 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42\": container with ID starting with fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42 not found: ID does not exist" containerID="fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42" Jul 14 11:36:40.159956 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.159916 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42"} err="failed to get container status \"fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42\": rpc error: code = NotFound desc = could not find container \"fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42\": container with ID starting with fa2304cc911a61f34f61c069b0352fc91cd583918fc18cfffe15fc0327188b42 not found: ID does not exist" Jul 14 11:36:40.171260 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.171234 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j"] Jul 14 11:36:40.174740 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:40.174719 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1cf7269b-75d7d76498-qqs5j"] Jul 14 11:36:41.491568 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:41.491495 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d31180f4-2ac3-4df1-81e8-1a01edf96f73" path="/var/lib/kubelet/pods/d31180f4-2ac3-4df1-81e8-1a01edf96f73/volumes" Jul 14 11:36:50.649531 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:50.649495 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:36:50.652840 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:50.652816 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:36:50.653611 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:50.653593 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:36:50.656752 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:36:50.656733 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:38:34.905483 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.905402 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c"] Jul 14 11:38:34.905964 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.905947 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d31180f4-2ac3-4df1-81e8-1a01edf96f73" containerName="maas-api" Jul 14 11:38:34.906010 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.905968 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d31180f4-2ac3-4df1-81e8-1a01edf96f73" containerName="maas-api" Jul 14 11:38:34.906071 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.906059 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d31180f4-2ac3-4df1-81e8-1a01edf96f73" containerName="maas-api" Jul 14 11:38:34.909529 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.909504 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:34.911620 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.911600 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1ba6ef6c-serving-cert\"" Jul 14 11:38:34.919620 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:34.919597 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c"] Jul 14 11:38:35.052830 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.052796 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d52a8f3-a24d-41a9-b510-4416f70e2f16-openshift-service-ca\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.052830 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.052831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwq58\" (UniqueName: \"kubernetes.io/projected/6d52a8f3-a24d-41a9-b510-4416f70e2f16-kube-api-access-fwq58\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.053027 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.052892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d52a8f3-a24d-41a9-b510-4416f70e2f16-maas-api-tls\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.153568 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.153539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d52a8f3-a24d-41a9-b510-4416f70e2f16-openshift-service-ca\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.153726 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.153575 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwq58\" (UniqueName: \"kubernetes.io/projected/6d52a8f3-a24d-41a9-b510-4416f70e2f16-kube-api-access-fwq58\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.153726 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.153610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d52a8f3-a24d-41a9-b510-4416f70e2f16-maas-api-tls\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.154295 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.154269 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d52a8f3-a24d-41a9-b510-4416f70e2f16-openshift-service-ca\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.156070 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.156021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d52a8f3-a24d-41a9-b510-4416f70e2f16-maas-api-tls\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.166567 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.166543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwq58\" (UniqueName: \"kubernetes.io/projected/6d52a8f3-a24d-41a9-b510-4416f70e2f16-kube-api-access-fwq58\") pod \"maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.221084 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.221056 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:35.372320 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.372292 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c"] Jul 14 11:38:35.374265 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:38:35.374241 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d52a8f3_a24d_41a9_b510_4416f70e2f16.slice/crio-409fff23d80808470e2bda15a8548df7ceca480e51657cee19757eb8f6cd315d WatchSource:0}: Error finding container 409fff23d80808470e2bda15a8548df7ceca480e51657cee19757eb8f6cd315d: Status 404 returned error can't find the container with id 409fff23d80808470e2bda15a8548df7ceca480e51657cee19757eb8f6cd315d Jul 14 11:38:35.375577 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.375562 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:38:35.714208 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:35.714176 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" event={"ID":"6d52a8f3-a24d-41a9-b510-4416f70e2f16","Type":"ContainerStarted","Data":"409fff23d80808470e2bda15a8548df7ceca480e51657cee19757eb8f6cd315d"} Jul 14 11:38:36.720143 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:36.720098 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" event={"ID":"6d52a8f3-a24d-41a9-b510-4416f70e2f16","Type":"ContainerStarted","Data":"3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef"} Jul 14 11:38:36.720537 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:36.720247 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:36.740498 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:36.740455 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" podStartSLOduration=2.299493407 podStartE2EDuration="2.740440915s" totalImagesPullingTime="440.947508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:38:36.738593279 +0000 UTC m=+1907.721564116" watchObservedRunningTime="2026-07-14 11:38:36.740440915 +0000 UTC m=+1907.723411750" Jul 14 11:38:42.730576 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:42.730547 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:57.211061 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.211031 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c"] Jul 14 11:38:57.211541 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.211291 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" podUID="6d52a8f3-a24d-41a9-b510-4416f70e2f16" containerName="maas-api" containerID="cri-o://3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef" gracePeriod=30 Jul 14 11:38:57.457470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.457442 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:57.529197 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.529075 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d52a8f3-a24d-41a9-b510-4416f70e2f16-openshift-service-ca\") pod \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " Jul 14 11:38:57.529197 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.529149 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d52a8f3-a24d-41a9-b510-4416f70e2f16-maas-api-tls\") pod \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " Jul 14 11:38:57.529419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.529237 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwq58\" (UniqueName: \"kubernetes.io/projected/6d52a8f3-a24d-41a9-b510-4416f70e2f16-kube-api-access-fwq58\") pod \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\" (UID: \"6d52a8f3-a24d-41a9-b510-4416f70e2f16\") " Jul 14 11:38:57.529507 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.529482 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d52a8f3-a24d-41a9-b510-4416f70e2f16-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d52a8f3-a24d-41a9-b510-4416f70e2f16" (UID: "6d52a8f3-a24d-41a9-b510-4416f70e2f16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:38:57.531045 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.531026 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d52a8f3-a24d-41a9-b510-4416f70e2f16-kube-api-access-fwq58" (OuterVolumeSpecName: "kube-api-access-fwq58") pod "6d52a8f3-a24d-41a9-b510-4416f70e2f16" (UID: "6d52a8f3-a24d-41a9-b510-4416f70e2f16"). InnerVolumeSpecName "kube-api-access-fwq58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:38:57.531153 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.531131 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d52a8f3-a24d-41a9-b510-4416f70e2f16-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d52a8f3-a24d-41a9-b510-4416f70e2f16" (UID: "6d52a8f3-a24d-41a9-b510-4416f70e2f16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:38:57.630674 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.630650 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d52a8f3-a24d-41a9-b510-4416f70e2f16-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:38:57.630674 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.630670 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d52a8f3-a24d-41a9-b510-4416f70e2f16-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:38:57.630807 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.630680 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwq58\" (UniqueName: \"kubernetes.io/projected/6d52a8f3-a24d-41a9-b510-4416f70e2f16-kube-api-access-fwq58\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:38:57.820938 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.820853 2581 generic.go:356] "Generic (PLEG): container finished" podID="6d52a8f3-a24d-41a9-b510-4416f70e2f16" containerID="3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef" exitCode=0 Jul 14 11:38:57.820938 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.820906 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" Jul 14 11:38:57.821210 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.820940 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" event={"ID":"6d52a8f3-a24d-41a9-b510-4416f70e2f16","Type":"ContainerDied","Data":"3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef"} Jul 14 11:38:57.821210 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.820995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c" event={"ID":"6d52a8f3-a24d-41a9-b510-4416f70e2f16","Type":"ContainerDied","Data":"409fff23d80808470e2bda15a8548df7ceca480e51657cee19757eb8f6cd315d"} Jul 14 11:38:57.821210 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.821024 2581 scope.go:117] "RemoveContainer" containerID="3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef" Jul 14 11:38:57.830625 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.830611 2581 scope.go:117] "RemoveContainer" containerID="3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef" Jul 14 11:38:57.830868 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:38:57.830849 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef\": container with ID starting with 3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef not found: ID does not exist" containerID="3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef" Jul 14 11:38:57.830912 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.830876 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef"} err="failed to get container status \"3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef\": rpc error: code = NotFound desc = could not find container \"3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef\": container with ID starting with 3619bce7c5bdade08432fec4938977d908cf2019f56b21e8903fc977fc0299ef not found: ID does not exist" Jul 14 11:38:57.843529 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.843510 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c"] Jul 14 11:38:57.847212 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:57.847193 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ba6ef6c-5886bb85b8-q8r7c"] Jul 14 11:38:59.492097 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:38:59.492067 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d52a8f3-a24d-41a9-b510-4416f70e2f16" path="/var/lib/kubelet/pods/6d52a8f3-a24d-41a9-b510-4416f70e2f16/volumes" Jul 14 11:39:35.708294 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.708263 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5"] Jul 14 11:39:35.708853 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.708671 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d52a8f3-a24d-41a9-b510-4416f70e2f16" containerName="maas-api" Jul 14 11:39:35.708853 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.708696 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d52a8f3-a24d-41a9-b510-4416f70e2f16" containerName="maas-api" Jul 14 11:39:35.708853 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.708822 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d52a8f3-a24d-41a9-b510-4416f70e2f16" containerName="maas-api" Jul 14 11:39:35.711454 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.711439 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.714769 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.714748 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c5ce9ebd-serving-cert\"" Jul 14 11:39:35.724329 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.724308 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5"] Jul 14 11:39:35.829772 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.829739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmr9d\" (UniqueName: \"kubernetes.io/projected/3874c91c-1672-432c-b3ce-0335091b412f-kube-api-access-mmr9d\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.829900 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.829814 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3874c91c-1672-432c-b3ce-0335091b412f-openshift-service-ca\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.829900 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.829884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3874c91c-1672-432c-b3ce-0335091b412f-maas-api-tls\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.931212 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.931183 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3874c91c-1672-432c-b3ce-0335091b412f-maas-api-tls\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.931358 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.931234 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmr9d\" (UniqueName: \"kubernetes.io/projected/3874c91c-1672-432c-b3ce-0335091b412f-kube-api-access-mmr9d\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.931358 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.931280 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3874c91c-1672-432c-b3ce-0335091b412f-openshift-service-ca\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.931961 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.931943 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3874c91c-1672-432c-b3ce-0335091b412f-openshift-service-ca\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.933446 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.933431 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3874c91c-1672-432c-b3ce-0335091b412f-maas-api-tls\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:35.940776 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:35.940754 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmr9d\" (UniqueName: \"kubernetes.io/projected/3874c91c-1672-432c-b3ce-0335091b412f-kube-api-access-mmr9d\") pod \"maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:36.020814 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:36.020757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:36.173332 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:36.173303 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5"] Jul 14 11:39:36.174458 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:39:36.174433 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3874c91c_1672_432c_b3ce_0335091b412f.slice/crio-5a0c5262ace55bf5e52a18c69549f1a1f5dd7fc549be11ec4c575b4b2744a7aa WatchSource:0}: Error finding container 5a0c5262ace55bf5e52a18c69549f1a1f5dd7fc549be11ec4c575b4b2744a7aa: Status 404 returned error can't find the container with id 5a0c5262ace55bf5e52a18c69549f1a1f5dd7fc549be11ec4c575b4b2744a7aa Jul 14 11:39:37.012999 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:37.012967 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" event={"ID":"3874c91c-1672-432c-b3ce-0335091b412f","Type":"ContainerStarted","Data":"2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4"} Jul 14 11:39:37.013390 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:37.013023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" event={"ID":"3874c91c-1672-432c-b3ce-0335091b412f","Type":"ContainerStarted","Data":"5a0c5262ace55bf5e52a18c69549f1a1f5dd7fc549be11ec4c575b4b2744a7aa"} Jul 14 11:39:37.013390 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:37.013050 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:39:37.033375 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:37.033326 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" podStartSLOduration=1.701157948 podStartE2EDuration="2.033311126s" totalImagesPullingTime="332.153178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:39:37.031451515 +0000 UTC m=+1968.014422360" watchObservedRunningTime="2026-07-14 11:39:37.033311126 +0000 UTC m=+1968.016281961" Jul 14 11:39:43.022883 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:39:43.022857 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:40:39.257220 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.257190 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5"] Jul 14 11:40:39.257716 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.257399 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" podUID="3874c91c-1672-432c-b3ce-0335091b412f" containerName="maas-api" containerID="cri-o://2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4" gracePeriod=30 Jul 14 11:40:39.505340 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.505318 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:40:39.619187 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.619102 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3874c91c-1672-432c-b3ce-0335091b412f-maas-api-tls\") pod \"3874c91c-1672-432c-b3ce-0335091b412f\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " Jul 14 11:40:39.619187 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.619150 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3874c91c-1672-432c-b3ce-0335091b412f-openshift-service-ca\") pod \"3874c91c-1672-432c-b3ce-0335091b412f\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " Jul 14 11:40:39.619349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.619243 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmr9d\" (UniqueName: \"kubernetes.io/projected/3874c91c-1672-432c-b3ce-0335091b412f-kube-api-access-mmr9d\") pod \"3874c91c-1672-432c-b3ce-0335091b412f\" (UID: \"3874c91c-1672-432c-b3ce-0335091b412f\") " Jul 14 11:40:39.619539 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.619514 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3874c91c-1672-432c-b3ce-0335091b412f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3874c91c-1672-432c-b3ce-0335091b412f" (UID: "3874c91c-1672-432c-b3ce-0335091b412f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:40:39.620911 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.620881 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3874c91c-1672-432c-b3ce-0335091b412f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3874c91c-1672-432c-b3ce-0335091b412f" (UID: "3874c91c-1672-432c-b3ce-0335091b412f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:40:39.620996 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.620967 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3874c91c-1672-432c-b3ce-0335091b412f-kube-api-access-mmr9d" (OuterVolumeSpecName: "kube-api-access-mmr9d") pod "3874c91c-1672-432c-b3ce-0335091b412f" (UID: "3874c91c-1672-432c-b3ce-0335091b412f"). InnerVolumeSpecName "kube-api-access-mmr9d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:40:39.720517 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.720496 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmr9d\" (UniqueName: \"kubernetes.io/projected/3874c91c-1672-432c-b3ce-0335091b412f-kube-api-access-mmr9d\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:40:39.720517 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.720517 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3874c91c-1672-432c-b3ce-0335091b412f-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:40:39.720666 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:39.720527 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3874c91c-1672-432c-b3ce-0335091b412f-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:40:40.316283 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.316251 2581 generic.go:356] "Generic (PLEG): container finished" podID="3874c91c-1672-432c-b3ce-0335091b412f" containerID="2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4" exitCode=0 Jul 14 11:40:40.316682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.316290 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" Jul 14 11:40:40.316682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.316300 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" event={"ID":"3874c91c-1672-432c-b3ce-0335091b412f","Type":"ContainerDied","Data":"2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4"} Jul 14 11:40:40.316682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.316339 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5" event={"ID":"3874c91c-1672-432c-b3ce-0335091b412f","Type":"ContainerDied","Data":"5a0c5262ace55bf5e52a18c69549f1a1f5dd7fc549be11ec4c575b4b2744a7aa"} Jul 14 11:40:40.316682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.316360 2581 scope.go:117] "RemoveContainer" containerID="2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4" Jul 14 11:40:40.325388 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.325368 2581 scope.go:117] "RemoveContainer" containerID="2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4" Jul 14 11:40:40.325623 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:40:40.325604 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4\": container with ID starting with 2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4 not found: ID does not exist" containerID="2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4" Jul 14 11:40:40.325688 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.325634 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4"} err="failed to get container status \"2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4\": rpc error: code = NotFound desc = could not find container \"2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4\": container with ID starting with 2cb1d43ef82e4ff2a2b335534c083ffadda5ad2a6a56a7473825a677e83477a4 not found: ID does not exist" Jul 14 11:40:40.336785 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.336763 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5"] Jul 14 11:40:40.340482 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:40.340461 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c5ce9ebd-b7556798d-8z9w5"] Jul 14 11:40:41.497026 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:40:41.496988 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3874c91c-1672-432c-b3ce-0335091b412f" path="/var/lib/kubelet/pods/3874c91c-1672-432c-b3ce-0335091b412f/volumes" Jul 14 11:41:50.681986 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:41:50.681892 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:41:50.685338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:41:50.684934 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:41:50.688090 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:41:50.688073 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:41:50.691250 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:41:50.691234 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:42:54.629913 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.629879 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg"] Jul 14 11:42:54.630357 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.630323 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3874c91c-1672-432c-b3ce-0335091b412f" containerName="maas-api" Jul 14 11:42:54.630357 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.630338 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3874c91c-1672-432c-b3ce-0335091b412f" containerName="maas-api" Jul 14 11:42:54.630432 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.630424 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3874c91c-1672-432c-b3ce-0335091b412f" containerName="maas-api" Jul 14 11:42:54.633483 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.633469 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.635329 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.635311 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1b0afa-serving-cert\"" Jul 14 11:42:54.643203 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.643181 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg"] Jul 14 11:42:54.781904 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.781865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b60d86-6103-4fe5-9b7d-fbbf65216a21-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.781904 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.781906 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.782100 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.781953 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk58m\" (UniqueName: \"kubernetes.io/projected/83b60d86-6103-4fe5-9b7d-fbbf65216a21-kube-api-access-xk58m\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.882547 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.882474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk58m\" (UniqueName: \"kubernetes.io/projected/83b60d86-6103-4fe5-9b7d-fbbf65216a21-kube-api-access-xk58m\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.882547 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.882537 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b60d86-6103-4fe5-9b7d-fbbf65216a21-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.882721 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.882555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.882721 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:42:54.882647 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-serving-cert: secret "maas-api-e2e-inf-a-1b0afa-serving-cert" not found Jul 14 11:42:54.882805 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:42:54.882722 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls podName:83b60d86-6103-4fe5-9b7d-fbbf65216a21 nodeName:}" failed. No retries permitted until 2026-07-14 11:42:55.382706751 +0000 UTC m=+2166.365677564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls") pod "maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" (UID: "83b60d86-6103-4fe5-9b7d-fbbf65216a21") : secret "maas-api-e2e-inf-a-1b0afa-serving-cert" not found Jul 14 11:42:54.883156 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.883110 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b60d86-6103-4fe5-9b7d-fbbf65216a21-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:54.893992 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:54.893972 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk58m\" (UniqueName: \"kubernetes.io/projected/83b60d86-6103-4fe5-9b7d-fbbf65216a21-kube-api-access-xk58m\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:55.386211 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:55.386174 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:55.388491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:55.388463 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls\") pod \"maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:55.544349 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:55.544323 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:55.675445 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:55.675422 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg"] Jul 14 11:42:55.677537 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:42:55.677509 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b60d86_6103_4fe5_9b7d_fbbf65216a21.slice/crio-b701ed0b07438da100f5d1b1cb20bdbdcd4230f19183f567aa55fcdb7c43b916 WatchSource:0}: Error finding container b701ed0b07438da100f5d1b1cb20bdbdcd4230f19183f567aa55fcdb7c43b916: Status 404 returned error can't find the container with id b701ed0b07438da100f5d1b1cb20bdbdcd4230f19183f567aa55fcdb7c43b916 Jul 14 11:42:55.962097 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:55.962020 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" event={"ID":"83b60d86-6103-4fe5-9b7d-fbbf65216a21","Type":"ContainerStarted","Data":"b701ed0b07438da100f5d1b1cb20bdbdcd4230f19183f567aa55fcdb7c43b916"} Jul 14 11:42:56.969359 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:56.969322 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" event={"ID":"83b60d86-6103-4fe5-9b7d-fbbf65216a21","Type":"ContainerStarted","Data":"4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818"} Jul 14 11:42:56.969702 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:56.969404 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:42:56.987482 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:42:56.987432 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" podStartSLOduration=2.596861378 podStartE2EDuration="2.987417006s" totalImagesPullingTime="390.555628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:42:56.98587719 +0000 UTC m=+2167.968848028" watchObservedRunningTime="2026-07-14 11:42:56.987417006 +0000 UTC m=+2167.970387840" Jul 14 11:43:02.809365 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.809326 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9"] Jul 14 11:43:02.812897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.812881 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.815790 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.815765 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f85160-serving-cert\"" Jul 14 11:43:02.830525 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.830500 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9"] Jul 14 11:43:02.848107 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.848078 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687b9f82-9973-4c04-8f6e-6b65cca369c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.848239 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.848147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687b9f82-9973-4c04-8f6e-6b65cca369c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.848239 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.848181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgwz5\" (UniqueName: \"kubernetes.io/projected/687b9f82-9973-4c04-8f6e-6b65cca369c9-kube-api-access-mgwz5\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.949397 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.949371 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687b9f82-9973-4c04-8f6e-6b65cca369c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.949505 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.949407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687b9f82-9973-4c04-8f6e-6b65cca369c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.949505 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.949428 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgwz5\" (UniqueName: \"kubernetes.io/projected/687b9f82-9973-4c04-8f6e-6b65cca369c9-kube-api-access-mgwz5\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.950004 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.949983 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687b9f82-9973-4c04-8f6e-6b65cca369c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.952370 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.952350 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687b9f82-9973-4c04-8f6e-6b65cca369c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.959100 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.959072 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgwz5\" (UniqueName: \"kubernetes.io/projected/687b9f82-9973-4c04-8f6e-6b65cca369c9-kube-api-access-mgwz5\") pod \"maas-api-e2e-inf-b-f85160-7795d88877-5wsv9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:02.986301 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:02.986277 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:43:03.124012 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:03.123984 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:03.260437 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:03.260411 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9"] Jul 14 11:43:03.262332 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:43:03.262304 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687b9f82_9973_4c04_8f6e_6b65cca369c9.slice/crio-01372d92f5d908c2b242b73e6884002bba1984d5223edf90d19114922ea11a75 WatchSource:0}: Error finding container 01372d92f5d908c2b242b73e6884002bba1984d5223edf90d19114922ea11a75: Status 404 returned error can't find the container with id 01372d92f5d908c2b242b73e6884002bba1984d5223edf90d19114922ea11a75 Jul 14 11:43:04.012807 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:04.012727 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" event={"ID":"687b9f82-9973-4c04-8f6e-6b65cca369c9","Type":"ContainerStarted","Data":"3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d"} Jul 14 11:43:04.012807 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:04.012779 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" event={"ID":"687b9f82-9973-4c04-8f6e-6b65cca369c9","Type":"ContainerStarted","Data":"01372d92f5d908c2b242b73e6884002bba1984d5223edf90d19114922ea11a75"} Jul 14 11:43:04.013239 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:04.012810 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:43:04.032083 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:04.032033 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" podStartSLOduration=1.6488060839999998 podStartE2EDuration="2.032016959s" totalImagesPullingTime="383.210875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:43:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:43:04.02899992 +0000 UTC m=+2175.011970754" watchObservedRunningTime="2026-07-14 11:43:04.032016959 +0000 UTC m=+2175.014987797" Jul 14 11:43:10.026668 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:43:10.026633 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:44:12.224241 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.224161 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg"] Jul 14 11:44:12.224793 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.224398 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" podUID="83b60d86-6103-4fe5-9b7d-fbbf65216a21" containerName="maas-api" containerID="cri-o://4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818" gracePeriod=30 Jul 14 11:44:12.668846 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.668818 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:44:12.728774 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.728745 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b60d86-6103-4fe5-9b7d-fbbf65216a21-openshift-service-ca\") pod \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " Jul 14 11:44:12.728924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.728798 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk58m\" (UniqueName: \"kubernetes.io/projected/83b60d86-6103-4fe5-9b7d-fbbf65216a21-kube-api-access-xk58m\") pod \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " Jul 14 11:44:12.728924 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.728888 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls\") pod \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\" (UID: \"83b60d86-6103-4fe5-9b7d-fbbf65216a21\") " Jul 14 11:44:12.729133 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.729092 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83b60d86-6103-4fe5-9b7d-fbbf65216a21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83b60d86-6103-4fe5-9b7d-fbbf65216a21" (UID: "83b60d86-6103-4fe5-9b7d-fbbf65216a21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:44:12.730799 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.730777 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83b60d86-6103-4fe5-9b7d-fbbf65216a21" (UID: "83b60d86-6103-4fe5-9b7d-fbbf65216a21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:44:12.730882 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.730817 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b60d86-6103-4fe5-9b7d-fbbf65216a21-kube-api-access-xk58m" (OuterVolumeSpecName: "kube-api-access-xk58m") pod "83b60d86-6103-4fe5-9b7d-fbbf65216a21" (UID: "83b60d86-6103-4fe5-9b7d-fbbf65216a21"). InnerVolumeSpecName "kube-api-access-xk58m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:44:12.829650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.829588 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk58m\" (UniqueName: \"kubernetes.io/projected/83b60d86-6103-4fe5-9b7d-fbbf65216a21-kube-api-access-xk58m\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:12.829650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.829612 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b60d86-6103-4fe5-9b7d-fbbf65216a21-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:12.829650 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:12.829626 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b60d86-6103-4fe5-9b7d-fbbf65216a21-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:13.364232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.364192 2581 generic.go:356] "Generic (PLEG): container finished" podID="83b60d86-6103-4fe5-9b7d-fbbf65216a21" containerID="4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818" exitCode=0 Jul 14 11:44:13.364654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.364240 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" Jul 14 11:44:13.364654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.364273 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" event={"ID":"83b60d86-6103-4fe5-9b7d-fbbf65216a21","Type":"ContainerDied","Data":"4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818"} Jul 14 11:44:13.364654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.364326 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg" event={"ID":"83b60d86-6103-4fe5-9b7d-fbbf65216a21","Type":"ContainerDied","Data":"b701ed0b07438da100f5d1b1cb20bdbdcd4230f19183f567aa55fcdb7c43b916"} Jul 14 11:44:13.364654 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.364348 2581 scope.go:117] "RemoveContainer" containerID="4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818" Jul 14 11:44:13.373939 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.373923 2581 scope.go:117] "RemoveContainer" containerID="4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818" Jul 14 11:44:13.374195 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:44:13.374177 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818\": container with ID starting with 4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818 not found: ID does not exist" containerID="4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818" Jul 14 11:44:13.374275 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.374206 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818"} err="failed to get container status \"4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818\": rpc error: code = NotFound desc = could not find container \"4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818\": container with ID starting with 4a0742974b20c0be453cc4a4eb6c200a41a99ae2c5a3fb363216bc6227ac4818 not found: ID does not exist" Jul 14 11:44:13.384322 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.384301 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg"] Jul 14 11:44:13.388251 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.388233 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1b0afa-77644b6d85-k89zg"] Jul 14 11:44:13.491816 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:13.491791 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b60d86-6103-4fe5-9b7d-fbbf65216a21" path="/var/lib/kubelet/pods/83b60d86-6103-4fe5-9b7d-fbbf65216a21/volumes" Jul 14 11:44:27.986872 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:27.986841 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf"] Jul 14 11:44:27.987322 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:27.987203 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b60d86-6103-4fe5-9b7d-fbbf65216a21" containerName="maas-api" Jul 14 11:44:27.987322 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:27.987214 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b60d86-6103-4fe5-9b7d-fbbf65216a21" containerName="maas-api" Jul 14 11:44:27.987322 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:27.987274 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b60d86-6103-4fe5-9b7d-fbbf65216a21" containerName="maas-api" Jul 14 11:44:27.990072 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:27.990050 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:27.992300 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:27.992278 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jtnsf\"" Jul 14 11:44:28.000861 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.000842 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf"] Jul 14 11:44:28.050555 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.050532 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eebcef3-d333-4a74-b1d8-f431c68c5280-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf\" (UID: \"8eebcef3-d333-4a74-b1d8-f431c68c5280\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:28.150931 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.150907 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eebcef3-d333-4a74-b1d8-f431c68c5280-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf\" (UID: \"8eebcef3-d333-4a74-b1d8-f431c68c5280\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:28.151468 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.151451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eebcef3-d333-4a74-b1d8-f431c68c5280-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf\" (UID: \"8eebcef3-d333-4a74-b1d8-f431c68c5280\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:28.302097 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.302027 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:28.428665 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.428630 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf"] Jul 14 11:44:28.432181 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:44:28.432156 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eebcef3_d333_4a74_b1d8_f431c68c5280.slice/crio-dedf43a5aec27ae74578f24c9a180a333f0efb9d356d1888faeec75264520cf4 WatchSource:0}: Error finding container dedf43a5aec27ae74578f24c9a180a333f0efb9d356d1888faeec75264520cf4: Status 404 returned error can't find the container with id dedf43a5aec27ae74578f24c9a180a333f0efb9d356d1888faeec75264520cf4 Jul 14 11:44:28.434035 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.434017 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:44:28.440661 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:28.440630 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" event={"ID":"8eebcef3-d333-4a74-b1d8-f431c68c5280","Type":"ContainerStarted","Data":"dedf43a5aec27ae74578f24c9a180a333f0efb9d356d1888faeec75264520cf4"} Jul 14 11:44:31.459317 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:31.459276 2581 generic.go:356] "Generic (PLEG): container finished" podID="8eebcef3-d333-4a74-b1d8-f431c68c5280" containerID="817dbd3968456fb9c27870e33d8ae2795db580761309303585149192ad37b606" exitCode=0 Jul 14 11:44:31.459684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:31.459365 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" event={"ID":"8eebcef3-d333-4a74-b1d8-f431c68c5280","Type":"ContainerDied","Data":"817dbd3968456fb9c27870e33d8ae2795db580761309303585149192ad37b606"} Jul 14 11:44:32.596200 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:32.596174 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:32.690449 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:32.690413 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eebcef3-d333-4a74-b1d8-f431c68c5280-maas-api-service-ca\") pod \"8eebcef3-d333-4a74-b1d8-f431c68c5280\" (UID: \"8eebcef3-d333-4a74-b1d8-f431c68c5280\") " Jul 14 11:44:32.690779 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:32.690758 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eebcef3-d333-4a74-b1d8-f431c68c5280-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8eebcef3-d333-4a74-b1d8-f431c68c5280" (UID: "8eebcef3-d333-4a74-b1d8-f431c68c5280"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:44:32.791186 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:32.791068 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eebcef3-d333-4a74-b1d8-f431c68c5280-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:33.470166 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:33.470134 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" Jul 14 11:44:33.470422 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:33.470166 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf" event={"ID":"8eebcef3-d333-4a74-b1d8-f431c68c5280","Type":"ContainerDied","Data":"dedf43a5aec27ae74578f24c9a180a333f0efb9d356d1888faeec75264520cf4"} Jul 14 11:44:33.470422 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:33.470218 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dedf43a5aec27ae74578f24c9a180a333f0efb9d356d1888faeec75264520cf4" Jul 14 11:44:38.419588 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:38.419553 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9"] Jul 14 11:44:38.420018 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:38.419761 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" podUID="687b9f82-9973-4c04-8f6e-6b65cca369c9" containerName="maas-api" containerID="cri-o://3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d" gracePeriod=30 Jul 14 11:44:39.665854 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.665834 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:44:39.747338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.747274 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgwz5\" (UniqueName: \"kubernetes.io/projected/687b9f82-9973-4c04-8f6e-6b65cca369c9-kube-api-access-mgwz5\") pod \"687b9f82-9973-4c04-8f6e-6b65cca369c9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " Jul 14 11:44:39.747338 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.747314 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687b9f82-9973-4c04-8f6e-6b65cca369c9-openshift-service-ca\") pod \"687b9f82-9973-4c04-8f6e-6b65cca369c9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " Jul 14 11:44:39.747537 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.747340 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687b9f82-9973-4c04-8f6e-6b65cca369c9-maas-api-tls\") pod \"687b9f82-9973-4c04-8f6e-6b65cca369c9\" (UID: \"687b9f82-9973-4c04-8f6e-6b65cca369c9\") " Jul 14 11:44:39.747712 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.747684 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687b9f82-9973-4c04-8f6e-6b65cca369c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "687b9f82-9973-4c04-8f6e-6b65cca369c9" (UID: "687b9f82-9973-4c04-8f6e-6b65cca369c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:44:39.749467 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.749444 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/687b9f82-9973-4c04-8f6e-6b65cca369c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "687b9f82-9973-4c04-8f6e-6b65cca369c9" (UID: "687b9f82-9973-4c04-8f6e-6b65cca369c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:44:39.749467 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.749455 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/687b9f82-9973-4c04-8f6e-6b65cca369c9-kube-api-access-mgwz5" (OuterVolumeSpecName: "kube-api-access-mgwz5") pod "687b9f82-9973-4c04-8f6e-6b65cca369c9" (UID: "687b9f82-9973-4c04-8f6e-6b65cca369c9"). InnerVolumeSpecName "kube-api-access-mgwz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:44:39.848528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.848505 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgwz5\" (UniqueName: \"kubernetes.io/projected/687b9f82-9973-4c04-8f6e-6b65cca369c9-kube-api-access-mgwz5\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:39.848528 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.848526 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687b9f82-9973-4c04-8f6e-6b65cca369c9-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:39.848661 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:39.848535 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687b9f82-9973-4c04-8f6e-6b65cca369c9-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:44:40.507267 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.507232 2581 generic.go:356] "Generic (PLEG): container finished" podID="687b9f82-9973-4c04-8f6e-6b65cca369c9" containerID="3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d" exitCode=0 Jul 14 11:44:40.507435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.507284 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" Jul 14 11:44:40.507435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.507314 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" event={"ID":"687b9f82-9973-4c04-8f6e-6b65cca369c9","Type":"ContainerDied","Data":"3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d"} Jul 14 11:44:40.507435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.507360 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9" event={"ID":"687b9f82-9973-4c04-8f6e-6b65cca369c9","Type":"ContainerDied","Data":"01372d92f5d908c2b242b73e6884002bba1984d5223edf90d19114922ea11a75"} Jul 14 11:44:40.507435 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.507383 2581 scope.go:117] "RemoveContainer" containerID="3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d" Jul 14 11:44:40.517024 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.517010 2581 scope.go:117] "RemoveContainer" containerID="3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d" Jul 14 11:44:40.517415 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:44:40.517392 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d\": container with ID starting with 3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d not found: ID does not exist" containerID="3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d" Jul 14 11:44:40.517517 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.517421 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d"} err="failed to get container status \"3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d\": rpc error: code = NotFound desc = could not find container \"3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d\": container with ID starting with 3b844cb045d9f656d9afc52039c3ad778cee966774ba3e3331a9c4ad45df8d9d not found: ID does not exist" Jul 14 11:44:40.529150 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.529110 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9"] Jul 14 11:44:40.534692 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:40.534669 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f85160-7795d88877-5wsv9"] Jul 14 11:44:41.491771 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:41.491735 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="687b9f82-9973-4c04-8f6e-6b65cca369c9" path="/var/lib/kubelet/pods/687b9f82-9973-4c04-8f6e-6b65cca369c9/volumes" Jul 14 11:44:58.404980 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.404945 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf"] Jul 14 11:44:58.405373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.405350 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eebcef3-d333-4a74-b1d8-f431c68c5280" containerName="revoke-keys" Jul 14 11:44:58.405373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.405363 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eebcef3-d333-4a74-b1d8-f431c68c5280" containerName="revoke-keys" Jul 14 11:44:58.405496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.405387 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="687b9f82-9973-4c04-8f6e-6b65cca369c9" containerName="maas-api" Jul 14 11:44:58.405496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.405396 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="687b9f82-9973-4c04-8f6e-6b65cca369c9" containerName="maas-api" Jul 14 11:44:58.405496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.405462 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="687b9f82-9973-4c04-8f6e-6b65cca369c9" containerName="maas-api" Jul 14 11:44:58.405496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.405480 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eebcef3-d333-4a74-b1d8-f431c68c5280" containerName="revoke-keys" Jul 14 11:44:58.408342 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.408327 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.410280 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.410261 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-bb21d4-serving-cert\"" Jul 14 11:44:58.417053 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.417030 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf"] Jul 14 11:44:58.500979 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.500878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.500979 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.500923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe6dc79-8e29-45c0-a04d-a528f063fb90-openshift-service-ca\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.501247 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.501015 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzlkt\" (UniqueName: \"kubernetes.io/projected/afe6dc79-8e29-45c0-a04d-a528f063fb90-kube-api-access-tzlkt\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.601631 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.601598 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.601759 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.601638 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe6dc79-8e29-45c0-a04d-a528f063fb90-openshift-service-ca\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.601759 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:44:58.601743 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-serving-cert: secret "maas-api-e2e-api-a-bb21d4-serving-cert" not found Jul 14 11:44:58.601881 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.601758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzlkt\" (UniqueName: \"kubernetes.io/projected/afe6dc79-8e29-45c0-a04d-a528f063fb90-kube-api-access-tzlkt\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.601881 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:44:58.601806 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls podName:afe6dc79-8e29-45c0-a04d-a528f063fb90 nodeName:}" failed. No retries permitted until 2026-07-14 11:44:59.101789783 +0000 UTC m=+2290.084760599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls") pod "maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" (UID: "afe6dc79-8e29-45c0-a04d-a528f063fb90") : secret "maas-api-e2e-api-a-bb21d4-serving-cert" not found Jul 14 11:44:58.602296 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.602278 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe6dc79-8e29-45c0-a04d-a528f063fb90-openshift-service-ca\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:58.612012 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:58.611991 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzlkt\" (UniqueName: \"kubernetes.io/projected/afe6dc79-8e29-45c0-a04d-a528f063fb90-kube-api-access-tzlkt\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:59.106635 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:59.106604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:59.108993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:59.108968 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls\") pod \"maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:59.320841 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:59.320806 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:44:59.655588 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:44:59.655561 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf"] Jul 14 11:44:59.657633 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:44:59.657608 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe6dc79_8e29_45c0_a04d_a528f063fb90.slice/crio-f726c8a9b3016e7c93a4da72c2fbdf3e9555037834339dd1fdd82a162bfaaa10 WatchSource:0}: Error finding container f726c8a9b3016e7c93a4da72c2fbdf3e9555037834339dd1fdd82a162bfaaa10: Status 404 returned error can't find the container with id f726c8a9b3016e7c93a4da72c2fbdf3e9555037834339dd1fdd82a162bfaaa10 Jul 14 11:45:00.605093 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:00.605008 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" event={"ID":"afe6dc79-8e29-45c0-a04d-a528f063fb90","Type":"ContainerStarted","Data":"df193197cc381ae1c1d887bb24372c3130b4598ab946ba6c0711fa606c0d1426"} Jul 14 11:45:00.605093 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:00.605090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" event={"ID":"afe6dc79-8e29-45c0-a04d-a528f063fb90","Type":"ContainerStarted","Data":"f726c8a9b3016e7c93a4da72c2fbdf3e9555037834339dd1fdd82a162bfaaa10"} Jul 14 11:45:00.605282 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:00.605164 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:45:00.623500 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:00.623454 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" podStartSLOduration=2.049347178 podStartE2EDuration="2.623438687s" totalImagesPullingTime="574.091509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:45:00.620380675 +0000 UTC m=+2291.603351509" watchObservedRunningTime="2026-07-14 11:45:00.623438687 +0000 UTC m=+2291.606409556" Jul 14 11:45:06.522628 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.522593 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2"] Jul 14 11:45:06.530599 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.530578 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.534356 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.534326 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-af8cc2-serving-cert\"" Jul 14 11:45:06.536332 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.536311 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2"] Jul 14 11:45:06.567842 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.567819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.567966 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.567854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkpgn\" (UniqueName: \"kubernetes.io/projected/a1da95c6-1cc9-4e94-ae81-50608994d8b1-kube-api-access-lkpgn\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.567966 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.567879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1da95c6-1cc9-4e94-ae81-50608994d8b1-openshift-service-ca\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.615280 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.615261 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:45:06.668705 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.668681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.668854 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.668716 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkpgn\" (UniqueName: \"kubernetes.io/projected/a1da95c6-1cc9-4e94-ae81-50608994d8b1-kube-api-access-lkpgn\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.668854 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.668735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1da95c6-1cc9-4e94-ae81-50608994d8b1-openshift-service-ca\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.668854 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:45:06.668826 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-serving-cert: secret "maas-api-e2e-api-b-af8cc2-serving-cert" not found Jul 14 11:45:06.669034 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:45:06.668903 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls podName:a1da95c6-1cc9-4e94-ae81-50608994d8b1 nodeName:}" failed. No retries permitted until 2026-07-14 11:45:07.168882018 +0000 UTC m=+2298.151852836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls") pod "maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" (UID: "a1da95c6-1cc9-4e94-ae81-50608994d8b1") : secret "maas-api-e2e-api-b-af8cc2-serving-cert" not found Jul 14 11:45:06.669414 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.669391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1da95c6-1cc9-4e94-ae81-50608994d8b1-openshift-service-ca\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:06.679855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:06.679834 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkpgn\" (UniqueName: \"kubernetes.io/projected/a1da95c6-1cc9-4e94-ae81-50608994d8b1-kube-api-access-lkpgn\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:07.173021 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:07.172985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:07.175399 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:07.175373 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls\") pod \"maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:07.444714 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:07.444631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:07.594831 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:07.594803 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2"] Jul 14 11:45:07.596491 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:45:07.596467 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1da95c6_1cc9_4e94_ae81_50608994d8b1.slice/crio-9f43e1f08eefce3c356785e2a0a4a62ac24a3e26c05fbbca6369fd1246006606 WatchSource:0}: Error finding container 9f43e1f08eefce3c356785e2a0a4a62ac24a3e26c05fbbca6369fd1246006606: Status 404 returned error can't find the container with id 9f43e1f08eefce3c356785e2a0a4a62ac24a3e26c05fbbca6369fd1246006606 Jul 14 11:45:07.644329 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:07.644301 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" event={"ID":"a1da95c6-1cc9-4e94-ae81-50608994d8b1","Type":"ContainerStarted","Data":"9f43e1f08eefce3c356785e2a0a4a62ac24a3e26c05fbbca6369fd1246006606"} Jul 14 11:45:08.650682 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:08.650648 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" event={"ID":"a1da95c6-1cc9-4e94-ae81-50608994d8b1","Type":"ContainerStarted","Data":"fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149"} Jul 14 11:45:08.651147 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:08.650764 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:08.668510 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:08.668466 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" podStartSLOduration=2.367909896 podStartE2EDuration="2.6684515s" totalImagesPullingTime="300.541604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:45:08.666016466 +0000 UTC m=+2299.648987426" watchObservedRunningTime="2026-07-14 11:45:08.6684515 +0000 UTC m=+2299.651422364" Jul 14 11:45:14.660297 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:14.660269 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:29.525380 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.525347 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf"] Jul 14 11:45:29.525945 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.525523 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" podUID="afe6dc79-8e29-45c0-a04d-a528f063fb90" containerName="maas-api" containerID="cri-o://df193197cc381ae1c1d887bb24372c3130b4598ab946ba6c0711fa606c0d1426" gracePeriod=30 Jul 14 11:45:29.756219 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.756188 2581 generic.go:356] "Generic (PLEG): container finished" podID="afe6dc79-8e29-45c0-a04d-a528f063fb90" containerID="df193197cc381ae1c1d887bb24372c3130b4598ab946ba6c0711fa606c0d1426" exitCode=0 Jul 14 11:45:29.756347 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.756246 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" event={"ID":"afe6dc79-8e29-45c0-a04d-a528f063fb90","Type":"ContainerDied","Data":"df193197cc381ae1c1d887bb24372c3130b4598ab946ba6c0711fa606c0d1426"} Jul 14 11:45:29.756347 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.756284 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" event={"ID":"afe6dc79-8e29-45c0-a04d-a528f063fb90","Type":"ContainerDied","Data":"f726c8a9b3016e7c93a4da72c2fbdf3e9555037834339dd1fdd82a162bfaaa10"} Jul 14 11:45:29.756347 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.756303 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f726c8a9b3016e7c93a4da72c2fbdf3e9555037834339dd1fdd82a162bfaaa10" Jul 14 11:45:29.769104 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.769084 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:45:29.836461 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.836399 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls\") pod \"afe6dc79-8e29-45c0-a04d-a528f063fb90\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " Jul 14 11:45:29.836584 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.836459 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzlkt\" (UniqueName: \"kubernetes.io/projected/afe6dc79-8e29-45c0-a04d-a528f063fb90-kube-api-access-tzlkt\") pod \"afe6dc79-8e29-45c0-a04d-a528f063fb90\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " Jul 14 11:45:29.836644 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.836627 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe6dc79-8e29-45c0-a04d-a528f063fb90-openshift-service-ca\") pod \"afe6dc79-8e29-45c0-a04d-a528f063fb90\" (UID: \"afe6dc79-8e29-45c0-a04d-a528f063fb90\") " Jul 14 11:45:29.836970 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.836941 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afe6dc79-8e29-45c0-a04d-a528f063fb90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "afe6dc79-8e29-45c0-a04d-a528f063fb90" (UID: "afe6dc79-8e29-45c0-a04d-a528f063fb90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:45:29.838465 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.838446 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe6dc79-8e29-45c0-a04d-a528f063fb90-kube-api-access-tzlkt" (OuterVolumeSpecName: "kube-api-access-tzlkt") pod "afe6dc79-8e29-45c0-a04d-a528f063fb90" (UID: "afe6dc79-8e29-45c0-a04d-a528f063fb90"). InnerVolumeSpecName "kube-api-access-tzlkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:45:29.838535 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.838463 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "afe6dc79-8e29-45c0-a04d-a528f063fb90" (UID: "afe6dc79-8e29-45c0-a04d-a528f063fb90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:45:29.938164 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.938145 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzlkt\" (UniqueName: \"kubernetes.io/projected/afe6dc79-8e29-45c0-a04d-a528f063fb90-kube-api-access-tzlkt\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:29.938164 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.938163 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afe6dc79-8e29-45c0-a04d-a528f063fb90-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:29.938324 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:29.938174 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afe6dc79-8e29-45c0-a04d-a528f063fb90-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:30.761523 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:30.761493 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf" Jul 14 11:45:30.783845 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:30.783817 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf"] Jul 14 11:45:30.788367 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:30.788341 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bb21d4-9cfdddf56-mgvbf"] Jul 14 11:45:31.491630 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:31.491597 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afe6dc79-8e29-45c0-a04d-a528f063fb90" path="/var/lib/kubelet/pods/afe6dc79-8e29-45c0-a04d-a528f063fb90/volumes" Jul 14 11:45:45.247968 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.247929 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b"] Jul 14 11:45:45.248395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.248305 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe6dc79-8e29-45c0-a04d-a528f063fb90" containerName="maas-api" Jul 14 11:45:45.248395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.248317 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe6dc79-8e29-45c0-a04d-a528f063fb90" containerName="maas-api" Jul 14 11:45:45.248395 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.248386 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe6dc79-8e29-45c0-a04d-a528f063fb90" containerName="maas-api" Jul 14 11:45:45.252730 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.252713 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:45.258239 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.258216 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jtnsf\"" Jul 14 11:45:45.272267 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.272247 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b"] Jul 14 11:45:45.353877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.353850 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b\" (UID: \"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:45.454858 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.454835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b\" (UID: \"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:45.455418 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.455402 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b\" (UID: \"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:45.562108 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.562045 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:45.690930 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.690906 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b"] Jul 14 11:45:45.692627 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:45:45.692597 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50a57ae_fc34_4b2f_bc18_d7acdc3bbe74.slice/crio-46461a1e38ded8617ceb55491127121ea1f913849b6bd04a4362ce6bee7f435a WatchSource:0}: Error finding container 46461a1e38ded8617ceb55491127121ea1f913849b6bd04a4362ce6bee7f435a: Status 404 returned error can't find the container with id 46461a1e38ded8617ceb55491127121ea1f913849b6bd04a4362ce6bee7f435a Jul 14 11:45:45.836534 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:45.836468 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" event={"ID":"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74","Type":"ContainerStarted","Data":"46461a1e38ded8617ceb55491127121ea1f913849b6bd04a4362ce6bee7f435a"} Jul 14 11:45:46.842878 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:46.842792 2581 generic.go:356] "Generic (PLEG): container finished" podID="f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74" containerID="239fd47f21431fa8e7ca7671e9d2bd8d988102a7c82690a4d677fb1a39fabb21" exitCode=0 Jul 14 11:45:46.843232 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:46.842871 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" event={"ID":"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74","Type":"ContainerDied","Data":"239fd47f21431fa8e7ca7671e9d2bd8d988102a7c82690a4d677fb1a39fabb21"} Jul 14 11:45:47.990545 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:47.990515 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:48.075506 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:48.075479 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74-maas-api-service-ca\") pod \"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74\" (UID: \"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74\") " Jul 14 11:45:48.075769 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:48.075748 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74" (UID: "f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:45:48.176035 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:48.176012 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:48.853070 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:48.853036 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" Jul 14 11:45:48.853070 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:48.853053 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b" event={"ID":"f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74","Type":"ContainerDied","Data":"46461a1e38ded8617ceb55491127121ea1f913849b6bd04a4362ce6bee7f435a"} Jul 14 11:45:48.853340 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:48.853103 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46461a1e38ded8617ceb55491127121ea1f913849b6bd04a4362ce6bee7f435a" Jul 14 11:45:55.795225 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:55.795173 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2"] Jul 14 11:45:55.795645 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:55.795426 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" podUID="a1da95c6-1cc9-4e94-ae81-50608994d8b1" containerName="maas-api" containerID="cri-o://fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149" gracePeriod=30 Jul 14 11:45:56.034380 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.034356 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:56.146989 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.146953 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1da95c6-1cc9-4e94-ae81-50608994d8b1-openshift-service-ca\") pod \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " Jul 14 11:45:56.147147 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.146999 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls\") pod \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " Jul 14 11:45:56.147147 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.147028 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkpgn\" (UniqueName: \"kubernetes.io/projected/a1da95c6-1cc9-4e94-ae81-50608994d8b1-kube-api-access-lkpgn\") pod \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\" (UID: \"a1da95c6-1cc9-4e94-ae81-50608994d8b1\") " Jul 14 11:45:56.147374 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.147347 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1da95c6-1cc9-4e94-ae81-50608994d8b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1da95c6-1cc9-4e94-ae81-50608994d8b1" (UID: "a1da95c6-1cc9-4e94-ae81-50608994d8b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:45:56.148974 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.148949 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1da95c6-1cc9-4e94-ae81-50608994d8b1-kube-api-access-lkpgn" (OuterVolumeSpecName: "kube-api-access-lkpgn") pod "a1da95c6-1cc9-4e94-ae81-50608994d8b1" (UID: "a1da95c6-1cc9-4e94-ae81-50608994d8b1"). InnerVolumeSpecName "kube-api-access-lkpgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:45:56.149050 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.148979 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1da95c6-1cc9-4e94-ae81-50608994d8b1" (UID: "a1da95c6-1cc9-4e94-ae81-50608994d8b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:45:56.248589 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.248565 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1da95c6-1cc9-4e94-ae81-50608994d8b1-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:56.248589 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.248588 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkpgn\" (UniqueName: \"kubernetes.io/projected/a1da95c6-1cc9-4e94-ae81-50608994d8b1-kube-api-access-lkpgn\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:56.248730 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.248599 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1da95c6-1cc9-4e94-ae81-50608994d8b1-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:45:56.893351 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.893309 2581 generic.go:356] "Generic (PLEG): container finished" podID="a1da95c6-1cc9-4e94-ae81-50608994d8b1" containerID="fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149" exitCode=0 Jul 14 11:45:56.893351 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.893352 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" Jul 14 11:45:56.893787 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.893373 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" event={"ID":"a1da95c6-1cc9-4e94-ae81-50608994d8b1","Type":"ContainerDied","Data":"fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149"} Jul 14 11:45:56.893787 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.893425 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2" event={"ID":"a1da95c6-1cc9-4e94-ae81-50608994d8b1","Type":"ContainerDied","Data":"9f43e1f08eefce3c356785e2a0a4a62ac24a3e26c05fbbca6369fd1246006606"} Jul 14 11:45:56.893787 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.893446 2581 scope.go:117] "RemoveContainer" containerID="fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149" Jul 14 11:45:56.904330 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.904316 2581 scope.go:117] "RemoveContainer" containerID="fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149" Jul 14 11:45:56.904564 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:45:56.904547 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149\": container with ID starting with fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149 not found: ID does not exist" containerID="fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149" Jul 14 11:45:56.904611 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.904572 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149"} err="failed to get container status \"fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149\": rpc error: code = NotFound desc = could not find container \"fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149\": container with ID starting with fc658117e3c6e963a33cc931ed4d039fbebdabe6947eb7d077fe9b377f74f149 not found: ID does not exist" Jul 14 11:45:56.918870 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.918845 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2"] Jul 14 11:45:56.923050 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:56.923025 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af8cc2-6fb4bd788f-ggvx2"] Jul 14 11:45:57.491670 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:45:57.491641 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1da95c6-1cc9-4e94-ae81-50608994d8b1" path="/var/lib/kubelet/pods/a1da95c6-1cc9-4e94-ae81-50608994d8b1/volumes" Jul 14 11:46:15.828818 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.828783 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm"] Jul 14 11:46:15.829344 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.829215 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1da95c6-1cc9-4e94-ae81-50608994d8b1" containerName="maas-api" Jul 14 11:46:15.829344 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.829228 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1da95c6-1cc9-4e94-ae81-50608994d8b1" containerName="maas-api" Jul 14 11:46:15.829344 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.829238 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74" containerName="revoke-keys" Jul 14 11:46:15.829344 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.829244 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74" containerName="revoke-keys" Jul 14 11:46:15.829344 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.829299 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1da95c6-1cc9-4e94-ae81-50608994d8b1" containerName="maas-api" Jul 14 11:46:15.829344 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.829307 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74" containerName="revoke-keys" Jul 14 11:46:15.833521 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.833499 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:15.836307 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.836289 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-6b8edc-serving-cert\"" Jul 14 11:46:15.847700 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:15.847674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm"] Jul 14 11:46:16.018456 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.018414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg8vt\" (UniqueName: \"kubernetes.io/projected/db2a50af-0053-46a6-a357-aae4dfeaf453-kube-api-access-dg8vt\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.018456 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.018458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2a50af-0053-46a6-a357-aae4dfeaf453-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.018659 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.018544 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.119436 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.119351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg8vt\" (UniqueName: \"kubernetes.io/projected/db2a50af-0053-46a6-a357-aae4dfeaf453-kube-api-access-dg8vt\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.119436 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.119394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2a50af-0053-46a6-a357-aae4dfeaf453-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.119666 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.119459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.119666 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:46:16.119614 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-serving-cert: secret "maas-api-e2e-shared-a-6b8edc-serving-cert" not found Jul 14 11:46:16.119768 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:46:16.119693 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls podName:db2a50af-0053-46a6-a357-aae4dfeaf453 nodeName:}" failed. No retries permitted until 2026-07-14 11:46:16.619671727 +0000 UTC m=+2367.602642546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls") pod "maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" (UID: "db2a50af-0053-46a6-a357-aae4dfeaf453") : secret "maas-api-e2e-shared-a-6b8edc-serving-cert" not found Jul 14 11:46:16.120067 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.120045 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2a50af-0053-46a6-a357-aae4dfeaf453-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.145442 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.145415 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg8vt\" (UniqueName: \"kubernetes.io/projected/db2a50af-0053-46a6-a357-aae4dfeaf453-kube-api-access-dg8vt\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.625374 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.625341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.627710 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.627687 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:16.744347 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:16.744317 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:17.100494 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:17.100470 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm"] Jul 14 11:46:17.101894 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:46:17.101867 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2a50af_0053_46a6_a357_aae4dfeaf453.slice/crio-b13bc8cda63f7577d724ebc8905f6adc17d474de640f81589a50277ddbd24cf4 WatchSource:0}: Error finding container b13bc8cda63f7577d724ebc8905f6adc17d474de640f81589a50277ddbd24cf4: Status 404 returned error can't find the container with id b13bc8cda63f7577d724ebc8905f6adc17d474de640f81589a50277ddbd24cf4 Jul 14 11:46:18.001957 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:18.001918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" event={"ID":"db2a50af-0053-46a6-a357-aae4dfeaf453","Type":"ContainerStarted","Data":"e0470c586ea228000c374d0bd5972cd14ae56dcb88eaf9c5793c6602bff69ac6"} Jul 14 11:46:18.002179 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:18.001974 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" event={"ID":"db2a50af-0053-46a6-a357-aae4dfeaf453","Type":"ContainerStarted","Data":"b13bc8cda63f7577d724ebc8905f6adc17d474de640f81589a50277ddbd24cf4"} Jul 14 11:46:18.002179 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:18.001997 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:18.038950 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:18.038900 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" podStartSLOduration=2.596038747 podStartE2EDuration="3.038884127s" totalImagesPullingTime="442.84538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:18.034982232 +0000 UTC m=+2369.017953065" watchObservedRunningTime="2026-07-14 11:46:18.038884127 +0000 UTC m=+2369.021854961" Jul 14 11:46:23.924505 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.924472 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd"] Jul 14 11:46:23.928134 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.928094 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:23.935183 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.935160 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-5c5a7d-serving-cert\"" Jul 14 11:46:23.960427 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.960403 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd"] Jul 14 11:46:23.986040 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.986014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c13e5ab9-f81d-492d-a891-a40577f72587-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:23.986194 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.986048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:23.986194 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:23.986156 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmdzq\" (UniqueName: \"kubernetes.io/projected/c13e5ab9-f81d-492d-a891-a40577f72587-kube-api-access-xmdzq\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.011660 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.011635 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:46:24.087258 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.087229 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.087441 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.087319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmdzq\" (UniqueName: \"kubernetes.io/projected/c13e5ab9-f81d-492d-a891-a40577f72587-kube-api-access-xmdzq\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.087441 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.087392 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c13e5ab9-f81d-492d-a891-a40577f72587-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.087619 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:46:24.087599 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-serving-cert: secret "maas-api-e2e-shared-b-5c5a7d-serving-cert" not found Jul 14 11:46:24.087675 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:46:24.087668 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls podName:c13e5ab9-f81d-492d-a891-a40577f72587 nodeName:}" failed. No retries permitted until 2026-07-14 11:46:24.58765082 +0000 UTC m=+2375.570621632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls") pod "maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" (UID: "c13e5ab9-f81d-492d-a891-a40577f72587") : secret "maas-api-e2e-shared-b-5c5a7d-serving-cert" not found Jul 14 11:46:24.087981 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.087964 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c13e5ab9-f81d-492d-a891-a40577f72587-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.109309 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.109287 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmdzq\" (UniqueName: \"kubernetes.io/projected/c13e5ab9-f81d-492d-a891-a40577f72587-kube-api-access-xmdzq\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.591018 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.590983 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.593335 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.593318 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls\") pod \"maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.837475 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.837447 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:24.971459 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:24.971437 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd"] Jul 14 11:46:24.973003 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:46:24.972972 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc13e5ab9_f81d_492d_a891_a40577f72587.slice/crio-58cf153fe4fc720b1f933514749cd8e8d97181f7ff72cf0b347c685d1ce02c77 WatchSource:0}: Error finding container 58cf153fe4fc720b1f933514749cd8e8d97181f7ff72cf0b347c685d1ce02c77: Status 404 returned error can't find the container with id 58cf153fe4fc720b1f933514749cd8e8d97181f7ff72cf0b347c685d1ce02c77 Jul 14 11:46:25.042176 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:25.042146 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" event={"ID":"c13e5ab9-f81d-492d-a891-a40577f72587","Type":"ContainerStarted","Data":"58cf153fe4fc720b1f933514749cd8e8d97181f7ff72cf0b347c685d1ce02c77"} Jul 14 11:46:26.048616 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:26.048579 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" event={"ID":"c13e5ab9-f81d-492d-a891-a40577f72587","Type":"ContainerStarted","Data":"912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07"} Jul 14 11:46:26.048991 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:26.048740 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:26.067163 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:26.067089 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" podStartSLOduration=2.719476473 podStartE2EDuration="3.067072153s" totalImagesPullingTime="347.59568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:26.065899803 +0000 UTC m=+2377.048870640" watchObservedRunningTime="2026-07-14 11:46:26.067072153 +0000 UTC m=+2377.050042986" Jul 14 11:46:32.059195 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:32.059166 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:46:50.716851 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:50.716742 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:46:50.737738 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:50.720055 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:46:50.737738 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:50.723889 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:46:50.737738 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:46:50.733182 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:51:49.707821 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:51:49.707750 2581 scope.go:117] "RemoveContainer" containerID="df193197cc381ae1c1d887bb24372c3130b4598ab946ba6c0711fa606c0d1426" Jul 14 11:51:50.749237 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:51:50.749142 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:51:50.754720 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:51:50.752413 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:51:50.766221 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:51:50.766199 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:51:50.769040 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:51:50.769025 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b2w2v_a0fa7ed7-bff7-4678-821d-60bd5ea115cd/ovn-acl-logging/0.log" Jul 14 11:52:21.127836 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.127805 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9"] Jul 14 11:52:21.131293 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.131277 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:21.134533 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.134518 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jtnsf\"" Jul 14 11:52:21.154166 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.154142 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9"] Jul 14 11:52:21.202106 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.202073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc39f87-8df0-4e0c-9377-e29dac0673a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9\" (UID: \"fdc39f87-8df0-4e0c-9377-e29dac0673a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:21.303433 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.303405 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc39f87-8df0-4e0c-9377-e29dac0673a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9\" (UID: \"fdc39f87-8df0-4e0c-9377-e29dac0673a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:21.303993 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.303969 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc39f87-8df0-4e0c-9377-e29dac0673a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9\" (UID: \"fdc39f87-8df0-4e0c-9377-e29dac0673a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:21.446703 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.446643 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:21.584572 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.584544 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9"] Jul 14 11:52:21.881172 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:21.881143 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" event={"ID":"fdc39f87-8df0-4e0c-9377-e29dac0673a4","Type":"ContainerStarted","Data":"4ff236422092d03e15c66044bdb3ccf08d71d78c57b420ba5df08c9b822138b8"} Jul 14 11:52:22.888340 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:22.888306 2581 generic.go:356] "Generic (PLEG): container finished" podID="fdc39f87-8df0-4e0c-9377-e29dac0673a4" containerID="cf91d318916e5dedf789343ae0dc2e46555fd546e2a29950bbd0707238256b97" exitCode=0 Jul 14 11:52:22.888703 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:22.888360 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" event={"ID":"fdc39f87-8df0-4e0c-9377-e29dac0673a4","Type":"ContainerDied","Data":"cf91d318916e5dedf789343ae0dc2e46555fd546e2a29950bbd0707238256b97"} Jul 14 11:52:24.022169 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.022142 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:24.128608 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.128579 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc39f87-8df0-4e0c-9377-e29dac0673a4-maas-api-service-ca\") pod \"fdc39f87-8df0-4e0c-9377-e29dac0673a4\" (UID: \"fdc39f87-8df0-4e0c-9377-e29dac0673a4\") " Jul 14 11:52:24.128906 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.128884 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc39f87-8df0-4e0c-9377-e29dac0673a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fdc39f87-8df0-4e0c-9377-e29dac0673a4" (UID: "fdc39f87-8df0-4e0c-9377-e29dac0673a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:52:24.229588 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.229528 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc39f87-8df0-4e0c-9377-e29dac0673a4-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:24.900403 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.900374 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" Jul 14 11:52:24.900403 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.900395 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9" event={"ID":"fdc39f87-8df0-4e0c-9377-e29dac0673a4","Type":"ContainerDied","Data":"4ff236422092d03e15c66044bdb3ccf08d71d78c57b420ba5df08c9b822138b8"} Jul 14 11:52:24.900660 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:24.900436 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ff236422092d03e15c66044bdb3ccf08d71d78c57b420ba5df08c9b822138b8" Jul 14 11:52:31.609306 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:31.609274 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm"] Jul 14 11:52:31.609753 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:31.609508 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" podUID="db2a50af-0053-46a6-a357-aae4dfeaf453" containerName="maas-api" containerID="cri-o://e0470c586ea228000c374d0bd5972cd14ae56dcb88eaf9c5793c6602bff69ac6" gracePeriod=30 Jul 14 11:52:31.942872 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:31.942842 2581 generic.go:356] "Generic (PLEG): container finished" podID="db2a50af-0053-46a6-a357-aae4dfeaf453" containerID="e0470c586ea228000c374d0bd5972cd14ae56dcb88eaf9c5793c6602bff69ac6" exitCode=0 Jul 14 11:52:31.943032 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:31.942911 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" event={"ID":"db2a50af-0053-46a6-a357-aae4dfeaf453","Type":"ContainerDied","Data":"e0470c586ea228000c374d0bd5972cd14ae56dcb88eaf9c5793c6602bff69ac6"} Jul 14 11:52:32.052228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.052206 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:52:32.197628 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.197537 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg8vt\" (UniqueName: \"kubernetes.io/projected/db2a50af-0053-46a6-a357-aae4dfeaf453-kube-api-access-dg8vt\") pod \"db2a50af-0053-46a6-a357-aae4dfeaf453\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " Jul 14 11:52:32.197784 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.197638 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2a50af-0053-46a6-a357-aae4dfeaf453-openshift-service-ca\") pod \"db2a50af-0053-46a6-a357-aae4dfeaf453\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " Jul 14 11:52:32.197784 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.197661 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls\") pod \"db2a50af-0053-46a6-a357-aae4dfeaf453\" (UID: \"db2a50af-0053-46a6-a357-aae4dfeaf453\") " Jul 14 11:52:32.198021 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.197999 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db2a50af-0053-46a6-a357-aae4dfeaf453-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db2a50af-0053-46a6-a357-aae4dfeaf453" (UID: "db2a50af-0053-46a6-a357-aae4dfeaf453"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:52:32.199759 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.199733 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db2a50af-0053-46a6-a357-aae4dfeaf453-kube-api-access-dg8vt" (OuterVolumeSpecName: "kube-api-access-dg8vt") pod "db2a50af-0053-46a6-a357-aae4dfeaf453" (UID: "db2a50af-0053-46a6-a357-aae4dfeaf453"). InnerVolumeSpecName "kube-api-access-dg8vt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:52:32.199884 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.199819 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db2a50af-0053-46a6-a357-aae4dfeaf453" (UID: "db2a50af-0053-46a6-a357-aae4dfeaf453"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:52:32.299217 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.299186 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2a50af-0053-46a6-a357-aae4dfeaf453-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:32.299217 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.299212 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2a50af-0053-46a6-a357-aae4dfeaf453-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:32.299217 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.299220 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg8vt\" (UniqueName: \"kubernetes.io/projected/db2a50af-0053-46a6-a357-aae4dfeaf453-kube-api-access-dg8vt\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:32.948801 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.948776 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" Jul 14 11:52:32.949228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.948819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm" event={"ID":"db2a50af-0053-46a6-a357-aae4dfeaf453","Type":"ContainerDied","Data":"b13bc8cda63f7577d724ebc8905f6adc17d474de640f81589a50277ddbd24cf4"} Jul 14 11:52:32.949228 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.948877 2581 scope.go:117] "RemoveContainer" containerID="e0470c586ea228000c374d0bd5972cd14ae56dcb88eaf9c5793c6602bff69ac6" Jul 14 11:52:32.977389 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.977361 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm"] Jul 14 11:52:32.982072 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:32.982049 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b8edc-66788fb76d-vg4bm"] Jul 14 11:52:33.492033 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:33.492002 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db2a50af-0053-46a6-a357-aae4dfeaf453" path="/var/lib/kubelet/pods/db2a50af-0053-46a6-a357-aae4dfeaf453/volumes" Jul 14 11:52:47.467905 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.467873 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265"] Jul 14 11:52:47.468373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.468251 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db2a50af-0053-46a6-a357-aae4dfeaf453" containerName="maas-api" Jul 14 11:52:47.468373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.468264 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db2a50af-0053-46a6-a357-aae4dfeaf453" containerName="maas-api" Jul 14 11:52:47.468373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.468275 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdc39f87-8df0-4e0c-9377-e29dac0673a4" containerName="revoke-keys" Jul 14 11:52:47.468373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.468280 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc39f87-8df0-4e0c-9377-e29dac0673a4" containerName="revoke-keys" Jul 14 11:52:47.468373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.468340 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdc39f87-8df0-4e0c-9377-e29dac0673a4" containerName="revoke-keys" Jul 14 11:52:47.468373 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.468350 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db2a50af-0053-46a6-a357-aae4dfeaf453" containerName="maas-api" Jul 14 11:52:47.472770 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.472753 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:47.476238 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.476221 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jtnsf\"" Jul 14 11:52:47.485848 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.485827 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265"] Jul 14 11:52:47.615227 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.615201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ce06ff-90d6-4216-a82b-a96bdf6657cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265\" (UID: \"08ce06ff-90d6-4216-a82b-a96bdf6657cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:47.716354 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.716327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ce06ff-90d6-4216-a82b-a96bdf6657cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265\" (UID: \"08ce06ff-90d6-4216-a82b-a96bdf6657cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:47.716910 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.716887 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ce06ff-90d6-4216-a82b-a96bdf6657cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265\" (UID: \"08ce06ff-90d6-4216-a82b-a96bdf6657cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:47.782580 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.782526 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:47.938570 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:47.938537 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265"] Jul 14 11:52:47.942063 ip-10-0-135-172 kubenswrapper[2581]: W0714 11:52:47.942035 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ce06ff_90d6_4216_a82b_a96bdf6657cf.slice/crio-b183f0043305682526ab02a25b00fe9c8246137e7cd1a6d8aa5ce3985c11eb4a WatchSource:0}: Error finding container b183f0043305682526ab02a25b00fe9c8246137e7cd1a6d8aa5ce3985c11eb4a: Status 404 returned error can't find the container with id b183f0043305682526ab02a25b00fe9c8246137e7cd1a6d8aa5ce3985c11eb4a Jul 14 11:52:48.035712 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:48.035651 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" event={"ID":"08ce06ff-90d6-4216-a82b-a96bdf6657cf","Type":"ContainerStarted","Data":"b183f0043305682526ab02a25b00fe9c8246137e7cd1a6d8aa5ce3985c11eb4a"} Jul 14 11:52:49.042946 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:49.042912 2581 generic.go:356] "Generic (PLEG): container finished" podID="08ce06ff-90d6-4216-a82b-a96bdf6657cf" containerID="66117bba8d90e2b2d0c101d17d30f877d75c790739ac82a7e404871e12e21597" exitCode=0 Jul 14 11:52:49.043341 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:49.042999 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" event={"ID":"08ce06ff-90d6-4216-a82b-a96bdf6657cf","Type":"ContainerDied","Data":"66117bba8d90e2b2d0c101d17d30f877d75c790739ac82a7e404871e12e21597"} Jul 14 11:52:50.174897 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:50.174874 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:50.335353 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:50.335273 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ce06ff-90d6-4216-a82b-a96bdf6657cf-maas-api-service-ca\") pod \"08ce06ff-90d6-4216-a82b-a96bdf6657cf\" (UID: \"08ce06ff-90d6-4216-a82b-a96bdf6657cf\") " Jul 14 11:52:50.335641 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:50.335619 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ce06ff-90d6-4216-a82b-a96bdf6657cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08ce06ff-90d6-4216-a82b-a96bdf6657cf" (UID: "08ce06ff-90d6-4216-a82b-a96bdf6657cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:52:50.436413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:50.436387 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ce06ff-90d6-4216-a82b-a96bdf6657cf-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:51.053748 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:51.053714 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" event={"ID":"08ce06ff-90d6-4216-a82b-a96bdf6657cf","Type":"ContainerDied","Data":"b183f0043305682526ab02a25b00fe9c8246137e7cd1a6d8aa5ce3985c11eb4a"} Jul 14 11:52:51.053948 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:51.053766 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b183f0043305682526ab02a25b00fe9c8246137e7cd1a6d8aa5ce3985c11eb4a" Jul 14 11:52:51.053948 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:51.053723 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265" Jul 14 11:52:58.017945 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.017907 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd"] Jul 14 11:52:58.018366 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.018143 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" podUID="c13e5ab9-f81d-492d-a891-a40577f72587" containerName="maas-api" containerID="cri-o://912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07" gracePeriod=30 Jul 14 11:52:58.255674 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.255651 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:52:58.404672 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.404644 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls\") pod \"c13e5ab9-f81d-492d-a891-a40577f72587\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " Jul 14 11:52:58.404843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.404747 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c13e5ab9-f81d-492d-a891-a40577f72587-openshift-service-ca\") pod \"c13e5ab9-f81d-492d-a891-a40577f72587\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " Jul 14 11:52:58.404843 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.404782 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmdzq\" (UniqueName: \"kubernetes.io/projected/c13e5ab9-f81d-492d-a891-a40577f72587-kube-api-access-xmdzq\") pod \"c13e5ab9-f81d-492d-a891-a40577f72587\" (UID: \"c13e5ab9-f81d-492d-a891-a40577f72587\") " Jul 14 11:52:58.405081 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.405060 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c13e5ab9-f81d-492d-a891-a40577f72587-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c13e5ab9-f81d-492d-a891-a40577f72587" (UID: "c13e5ab9-f81d-492d-a891-a40577f72587"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:52:58.406674 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.406653 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c13e5ab9-f81d-492d-a891-a40577f72587" (UID: "c13e5ab9-f81d-492d-a891-a40577f72587"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:52:58.406742 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.406721 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c13e5ab9-f81d-492d-a891-a40577f72587-kube-api-access-xmdzq" (OuterVolumeSpecName: "kube-api-access-xmdzq") pod "c13e5ab9-f81d-492d-a891-a40577f72587" (UID: "c13e5ab9-f81d-492d-a891-a40577f72587"). InnerVolumeSpecName "kube-api-access-xmdzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:52:58.505834 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.505812 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c13e5ab9-f81d-492d-a891-a40577f72587-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:58.505834 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.505833 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c13e5ab9-f81d-492d-a891-a40577f72587-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:58.506003 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:58.505843 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmdzq\" (UniqueName: \"kubernetes.io/projected/c13e5ab9-f81d-492d-a891-a40577f72587-kube-api-access-xmdzq\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\"" Jul 14 11:52:59.096491 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.096456 2581 generic.go:356] "Generic (PLEG): container finished" podID="c13e5ab9-f81d-492d-a891-a40577f72587" containerID="912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07" exitCode=0 Jul 14 11:52:59.096870 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.096501 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" Jul 14 11:52:59.096870 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.096515 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" event={"ID":"c13e5ab9-f81d-492d-a891-a40577f72587","Type":"ContainerDied","Data":"912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07"} Jul 14 11:52:59.096870 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.096566 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd" event={"ID":"c13e5ab9-f81d-492d-a891-a40577f72587","Type":"ContainerDied","Data":"58cf153fe4fc720b1f933514749cd8e8d97181f7ff72cf0b347c685d1ce02c77"} Jul 14 11:52:59.096870 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.096598 2581 scope.go:117] "RemoveContainer" containerID="912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07" Jul 14 11:52:59.105413 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.105394 2581 scope.go:117] "RemoveContainer" containerID="912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07" Jul 14 11:52:59.105667 ip-10-0-135-172 kubenswrapper[2581]: E0714 11:52:59.105648 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07\": container with ID starting with 912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07 not found: ID does not exist" containerID="912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07" Jul 14 11:52:59.105723 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.105676 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07"} err="failed to get container status \"912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07\": rpc error: code = NotFound desc = could not find container \"912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07\": container with ID starting with 912a8e1f93d4359d3664dd3a4c28775a8ec92a893d7988a6f28483add5382f07 not found: ID does not exist" Jul 14 11:52:59.120380 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.120348 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd"] Jul 14 11:52:59.122524 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.122506 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5c5a7d-75c4ddccd4-8kbnd"] Jul 14 11:52:59.492073 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:52:59.492045 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c13e5ab9-f81d-492d-a891-a40577f72587" path="/var/lib/kubelet/pods/c13e5ab9-f81d-492d-a891-a40577f72587/volumes" Jul 14 11:53:20.145198 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:20.145151 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-vrcv2_abbe1276-2227-4a1e-9ba0-02a72d128d63/copy-manifests/0.log" Jul 14 11:53:20.161270 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:20.161239 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-vrcv2_abbe1276-2227-4a1e-9ba0-02a72d128d63/manager/0.log" Jul 14 11:53:20.532447 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:20.532369 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c786b4db6-fflpp_edd94c02-7e09-4096-a998-651ae8c3aced/maas-api/0.log" Jul 14 11:53:21.275952 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:21.275924 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-af8cc2-nqg8b_f50a57ae-fc34-4b2f-bc18-d7acdc3bbe74/revoke-keys/0.log" Jul 14 11:53:21.643422 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:21.643393 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-f85160-kc2zf_8eebcef3-d333-4a74-b1d8-f431c68c5280/revoke-keys/0.log" Jul 14 11:53:22.123846 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:22.123818 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-6b8edc-k5zb9_fdc39f87-8df0-4e0c-9377-e29dac0673a4/revoke-keys/0.log" Jul 14 11:53:22.239423 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:22.239395 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-5c5a7d-r2265_08ce06ff-90d6-4216-a82b-a96bdf6657cf/revoke-keys/0.log" Jul 14 11:53:22.361570 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:22.361538 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-lkmbr_11542375-3325-45f2-9c0e-fdfec9f2dc39/postgres/0.log" Jul 14 11:53:23.196180 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.196144 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5_312d31bd-8ad4-4d96-9499-1f1dd2bd3223/util/0.log" Jul 14 11:53:23.203513 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.203488 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5_312d31bd-8ad4-4d96-9499-1f1dd2bd3223/pull/0.log" Jul 14 11:53:23.210273 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.210253 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5_312d31bd-8ad4-4d96-9499-1f1dd2bd3223/extract/0.log" Jul 14 11:53:23.331036 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.331009 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf_df0d193f-1f62-4aa7-b8f4-4a11600bb3a3/util/0.log" Jul 14 11:53:23.338699 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.338679 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf_df0d193f-1f62-4aa7-b8f4-4a11600bb3a3/pull/0.log" Jul 14 11:53:23.347559 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.347540 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf_df0d193f-1f62-4aa7-b8f4-4a11600bb3a3/extract/0.log" Jul 14 11:53:23.466289 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.466216 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v_8136bff3-e8b5-46f3-bd19-757cc93ba612/util/0.log" Jul 14 11:53:23.476270 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.476250 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v_8136bff3-e8b5-46f3-bd19-757cc93ba612/pull/0.log" Jul 14 11:53:23.483141 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.483107 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v_8136bff3-e8b5-46f3-bd19-757cc93ba612/extract/0.log" Jul 14 11:53:23.619005 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.618979 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw_dc49c2c0-6196-45ed-ba57-7d26852538bf/extract/0.log" Jul 14 11:53:23.626664 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.626642 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw_dc49c2c0-6196-45ed-ba57-7d26852538bf/util/0.log" Jul 14 11:53:23.634999 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:23.634986 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw_dc49c2c0-6196-45ed-ba57-7d26852538bf/pull/0.log" Jul 14 11:53:25.179433 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:25.179404 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x7gsr_7b2cfdab-75cd-49be-8d20-472434f8a0f4/discovery/0.log" Jul 14 11:53:26.393919 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:26.393889 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-pxfr9_4cfff449-d3ff-433c-a9df-6d3855008c22/main/0.log" Jul 14 11:53:26.636104 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:26.636078 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhdxlf_2978df22-5d81-4a08-823d-f2f6cf5a0fa9/main/0.log" Jul 14 11:53:26.760106 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:26.760032 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-f6r9j_754c0a11-67de-42aa-aaf8-1e68f21c0acc/main/0.log" Jul 14 11:53:33.861574 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:33.861532 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mwx66_60660683-0b39-4db6-a3ad-0261e7a7a741/global-pull-secret-syncer/0.log" Jul 14 11:53:34.032828 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:34.032790 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wlfnm_1b7bed04-1028-4f4d-8a65-03347daeef1a/konnectivity-agent/0.log" Jul 14 11:53:34.088306 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:34.088281 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-172.ec2.internal_9437156ea6019366c0c95106c22b0536/haproxy/0.log" Jul 14 11:53:38.752743 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.752700 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5_312d31bd-8ad4-4d96-9499-1f1dd2bd3223/extract/0.log" Jul 14 11:53:38.780059 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.780034 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5_312d31bd-8ad4-4d96-9499-1f1dd2bd3223/util/0.log" Jul 14 11:53:38.808731 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.808693 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nk2s5_312d31bd-8ad4-4d96-9499-1f1dd2bd3223/pull/0.log" Jul 14 11:53:38.847268 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.847236 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf_df0d193f-1f62-4aa7-b8f4-4a11600bb3a3/extract/0.log" Jul 14 11:53:38.880361 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.880339 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf_df0d193f-1f62-4aa7-b8f4-4a11600bb3a3/util/0.log" Jul 14 11:53:38.914132 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.914089 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dmglf_df0d193f-1f62-4aa7-b8f4-4a11600bb3a3/pull/0.log" Jul 14 11:53:38.955910 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.955890 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v_8136bff3-e8b5-46f3-bd19-757cc93ba612/extract/0.log" Jul 14 11:53:38.992125 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:38.992100 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v_8136bff3-e8b5-46f3-bd19-757cc93ba612/util/0.log" Jul 14 11:53:39.023705 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:39.023652 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqm4v_8136bff3-e8b5-46f3-bd19-757cc93ba612/pull/0.log" Jul 14 11:53:39.070429 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:39.070408 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw_dc49c2c0-6196-45ed-ba57-7d26852538bf/extract/0.log" Jul 14 11:53:39.099822 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:39.099800 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw_dc49c2c0-6196-45ed-ba57-7d26852538bf/util/0.log" Jul 14 11:53:39.134301 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:39.134279 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w6kkw_dc49c2c0-6196-45ed-ba57-7d26852538bf/pull/0.log" Jul 14 11:53:41.992257 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:41.992230 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-cf46g_6986c71b-b111-413a-95e7-30a00b1d2068/cluster-monitoring-operator/0.log" Jul 14 11:53:42.017908 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.017886 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-jg2xc_2c539d70-c00b-49c0-b121-3862b52eb80b/kube-state-metrics/0.log" Jul 14 11:53:42.046152 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.046110 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-jg2xc_2c539d70-c00b-49c0-b121-3862b52eb80b/kube-rbac-proxy-main/0.log" Jul 14 11:53:42.075918 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.075898 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-jg2xc_2c539d70-c00b-49c0-b121-3862b52eb80b/kube-rbac-proxy-self/0.log" Jul 14 11:53:42.112091 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.112074 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-54885f4554-wt9dp_fb8e69bc-5c50-4ccc-8dc0-91d55fbd9181/metrics-server/0.log" Jul 14 11:53:42.181214 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.181194 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-msl2k_6d3936f7-54ac-4ab3-bf2e-dba215e060a6/node-exporter/0.log" Jul 14 11:53:42.213698 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.213677 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-msl2k_6d3936f7-54ac-4ab3-bf2e-dba215e060a6/kube-rbac-proxy/0.log" Jul 14 11:53:42.252222 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.252170 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-msl2k_6d3936f7-54ac-4ab3-bf2e-dba215e060a6/init-textfile/0.log" Jul 14 11:53:42.958278 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.958247 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5445b8485d-gf7cz_54ef4e74-4697-462e-9348-0a64f3369b7d/thanos-query/0.log" Jul 14 11:53:42.983135 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:42.983085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5445b8485d-gf7cz_54ef4e74-4697-462e-9348-0a64f3369b7d/kube-rbac-proxy-web/0.log" Jul 14 11:53:43.006855 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.006834 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5445b8485d-gf7cz_54ef4e74-4697-462e-9348-0a64f3369b7d/kube-rbac-proxy/0.log" Jul 14 11:53:43.034023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.033995 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5445b8485d-gf7cz_54ef4e74-4697-462e-9348-0a64f3369b7d/prom-label-proxy/0.log" Jul 14 11:53:43.058597 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.058569 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5445b8485d-gf7cz_54ef4e74-4697-462e-9348-0a64f3369b7d/kube-rbac-proxy-rules/0.log" Jul 14 11:53:43.093330 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.093284 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5445b8485d-gf7cz_54ef4e74-4697-462e-9348-0a64f3369b7d/kube-rbac-proxy-metrics/0.log" Jul 14 11:53:43.543298 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543271 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v"] Jul 14 11:53:43.543591 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543580 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c13e5ab9-f81d-492d-a891-a40577f72587" containerName="maas-api" Jul 14 11:53:43.543638 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543593 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c13e5ab9-f81d-492d-a891-a40577f72587" containerName="maas-api" Jul 14 11:53:43.543638 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543602 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08ce06ff-90d6-4216-a82b-a96bdf6657cf" containerName="revoke-keys" Jul 14 11:53:43.543638 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543607 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ce06ff-90d6-4216-a82b-a96bdf6657cf" containerName="revoke-keys" Jul 14 11:53:43.543748 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543691 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08ce06ff-90d6-4216-a82b-a96bdf6657cf" containerName="revoke-keys" Jul 14 11:53:43.543748 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.543701 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c13e5ab9-f81d-492d-a891-a40577f72587" containerName="maas-api" Jul 14 11:53:43.546797 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.546776 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.552023 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.551998 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j46tr\"/\"default-dockercfg-ntl7c\"" Jul 14 11:53:43.552430 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.552415 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j46tr\"/\"openshift-service-ca.crt\"" Jul 14 11:53:43.556496 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.556478 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j46tr\"/\"kube-root-ca.crt\"" Jul 14 11:53:43.581832 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.581800 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v"] Jul 14 11:53:43.636877 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.636848 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-sys\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.637048 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.636886 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-proc\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.637048 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.636912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-podres\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.637048 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.637001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-lib-modules\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.637048 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.637038 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxb6r\" (UniqueName: \"kubernetes.io/projected/5f356614-b5dd-463d-8372-78f9f5bdaa71-kube-api-access-jxb6r\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737570 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-podres\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-lib-modules\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737628 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxb6r\" (UniqueName: \"kubernetes.io/projected/5f356614-b5dd-463d-8372-78f9f5bdaa71-kube-api-access-jxb6r\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-sys\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-proc\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737747 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737717 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-podres\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737937 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737753 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-sys\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737937 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737772 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-proc\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.737937 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.737785 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f356614-b5dd-463d-8372-78f9f5bdaa71-lib-modules\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.746167 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.746143 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxb6r\" (UniqueName: \"kubernetes.io/projected/5f356614-b5dd-463d-8372-78f9f5bdaa71-kube-api-access-jxb6r\") pod \"perf-node-gather-daemonset-lnk8v\" (UID: \"5f356614-b5dd-463d-8372-78f9f5bdaa71\") " pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.856447 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.856371 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:43.994514 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:43.993846 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v"] Jul 14 11:53:44.276684 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.276655 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-kczsz_82f556b1-d504-4d9d-98be-d3cfd0485a15/networking-console-plugin/0.log" Jul 14 11:53:44.338554 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.338525 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" event={"ID":"5f356614-b5dd-463d-8372-78f9f5bdaa71","Type":"ContainerStarted","Data":"7ab77bd1df507279c105c5c2bcafd3aeb5e7f0b4cc93f9d34aac67c614fc5c70"} Jul 14 11:53:44.338719 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.338572 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" event={"ID":"5f356614-b5dd-463d-8372-78f9f5bdaa71","Type":"ContainerStarted","Data":"c384413c509c9c8f42092cf2e141818022fb140577f2a95e82af08fbfffcd22a"} Jul 14 11:53:44.338719 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.338669 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:44.359472 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.359423 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" podStartSLOduration=1.359409983 podStartE2EDuration="1.359409983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:53:44.35733022 +0000 UTC m=+2815.340301056" watchObservedRunningTime="2026-07-14 11:53:44.359409983 +0000 UTC m=+2815.342380857" Jul 14 11:53:44.842189 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.842165 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/1.log" Jul 14 11:53:44.846997 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:44.846977 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hrpl5_bca091db-f43d-447d-beb1-24cb429d4aad/console-operator/2.log" Jul 14 11:53:47.199920 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:47.199890 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c7pwn_08a7f632-942c-4b96-893e-ce794f5b1932/dns/0.log" Jul 14 11:53:47.233470 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:47.233447 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c7pwn_08a7f632-942c-4b96-893e-ce794f5b1932/kube-rbac-proxy/0.log" Jul 14 11:53:47.442917 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:47.442891 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9rqsm_8e708d4a-7e79-4aae-b247-fbdea80ca0f5/dns-node-resolver/0.log" Jul 14 11:53:48.068668 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:48.068637 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6948cf4b79-hhcdx_8bd99376-af9d-4ec8-906e-82f00bbad8cc/registry/0.log" Jul 14 11:53:48.095989 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:48.095961 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9n7wv_8eb6e134-f251-4b16-8c61-9a8e76e30d4d/node-ca/0.log" Jul 14 11:53:49.281792 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:49.281765 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x7gsr_7b2cfdab-75cd-49be-8d20-472434f8a0f4/discovery/0.log" Jul 14 11:53:50.230812 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:50.230783 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fqw6w_e28243da-e892-4689-8556-41b7e019f242/serve-healthcheck-canary/0.log" Jul 14 11:53:50.352961 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:50.352935 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j46tr/perf-node-gather-daemonset-lnk8v" Jul 14 11:53:50.833188 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:50.833139 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7sf5x_091c0e73-d9d2-4892-8912-3b249e1dfcbb/kube-rbac-proxy/0.log" Jul 14 11:53:50.881969 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:50.881945 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7sf5x_091c0e73-d9d2-4892-8912-3b249e1dfcbb/exporter/0.log" Jul 14 11:53:50.935589 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:50.935570 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7sf5x_091c0e73-d9d2-4892-8912-3b249e1dfcbb/extractor/0.log" Jul 14 11:53:54.200419 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:54.200392 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-vrcv2_abbe1276-2227-4a1e-9ba0-02a72d128d63/manager/0.log" Jul 14 11:53:54.219480 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:54.219459 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff488f585-vrcv2_abbe1276-2227-4a1e-9ba0-02a72d128d63/copy-manifests/0.log" Jul 14 11:53:54.394343 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:54.394312 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x7gsr_7b2cfdab-75cd-49be-8d20-472434f8a0f4/discovery/0.log" Jul 14 11:53:55.529377 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:55.529341 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7d84b86c7f-jrsrz_af10a7fc-cda8-4954-b038-4627c4c8b59c/manager/0.log" Jul 14 11:53:55.596424 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:55.596394 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-lz6j9_ccf52dba-e59f-430b-9ff6-8618c1c2121e/openshift-lws-operator/0.log" Jul 14 11:53:55.913445 ip-10-0-135-172 kubenswrapper[2581]: I0714 11:53:55.913413 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x7gsr_7b2cfdab-75cd-49be-8d20-472434f8a0f4/discovery/0.log"